The historians mentioned that Yazeed was pleased with the killing of Al-Hussayn (peace be upon him) and those with him as well as the captivity of the family of the Messenger of God (may God’s prayers be upon him and his family). His happiness appeared in his council, not caring to show his atheism and his disbelief when he uttered the poetry of Ibn Zaba'ri, denying the revelation descending on the Messenger of God; Mohammed (Allah's prayers be upon him and his family), but many blamed him, and it became clear to him the failure and error in his act that was not committed even by those who did not impersonate the religion of Islam.
His close advisers, his household and his women blamed him for that, and he has seen with his own eyes and heard with his own ears the holy head speaking when he commanded the killing of the Messenger of the King of Rome (there is no strength or power but God), and also the echo of his heinous crime and extreme cruelty was strong in all parts of Damascus, so he found no escape from blaming Ibn Ziyad for it.
And when he feared sedition and the turn of the matter against him, he hastened the liberation of Imam as-Sajjad (peace be upon him) and the captives from the family of Imam al-Hussayn (peace be upon him) from the Levant and sending them back to their homeland, and enabled them to do what they wanted, and he ordered Nu'man bin Bashir and a group with him to walk with them to Madinah with kindness, and they stopped on their way in Karbala where Imam al-Hussayn (peace be upon him) and his family and companions (peace be upon them) were martyred and buried.
When Yazeed wanted to equip them, he said to Nu'man bin Bashir, the companion of the Messenger of God (Allah's prayers be upon him and upon his holy Household): Equip these women with what is suitable for them, and send a faithful and righteous man with them from the people of the Levant, and send horses and helpers with them.
Imam as-Sadeq (peace be upon him) said: When Imam Ali Ibn Al-Hussayn (peace be upon him) left the Levant with the people of his household and a group of servants whom were sent by Yazeed with the Ahl al-Bayt (peace be upon them), and on top of them was Nu'man Ibn Bashir al-Ansari. They brought with them the head of Al-Hussayn (peace be upon him) and the heads of the rest of his companions to bury them with their bodies. When Imam as-Sajjad (peace be upon him) reached Iraq, he told the guide: Pass by Karbala. When they reached Karbala, the heads were attached to their bodies, and Jaber [al-Ansari] was found at the grave of al-Hussayn (peace be upon him).