@Gupta-Anubhav12 @Amritpal2001 Thank you very much for taking on this task. It looks pretty clean now.

Just highlighting some potential minor issues.

• If you see the English translations, somehow most of them don't have a full stop (or other punctuations) at the end.

• Same for Hindi translations and commentaries.

Example:

What it looks like now:

हे द्विजोत्तम! हमारे पक्षमें भी जो मुख्य हैं, उनपर भी आप ध्यान दीजिये। आपको याद दिलानेके लिये मेरी सेनाके जो नायक हैं, उनको मैं कहता हूँ

What it should be like:

हे द्विजोत्तम! हमारे पक्ष में भी जो मुख्य हैं, उनपर भी आप ध्यान दीजिये। आपको याद दिलाने के लिये मेरी सेना के जो नायक हैं, उनको मैं कहता हूँ।

• For all the Sanskrit text, we have removed the verse number from the end along with the newline characters. Here's what a sample text looks like right now:

धृतराष्ट्र उवाच\n\nधर्मक्षेत्रे कुरुक्षेत्रे समवेता युयुत्सवः।\n\nमामकाः पाण्डवाश्चैव किमकुर्वत सञ्जय

Here's what it should look like:

धृतराष्ट्र उवाच |\n\nधर्मक्षेत्रे कुरुक्षेत्रे समवेता युयुत्सवः |\n\nमामकाः पाण्डवाश्चैव किमकुर्वत सञ्जय ||1||

I am not sure whether this directly comes from the data that we extracted or whether there is some issue with the cleaning script that seems to be somehow removing the punctuations when it removes the newline characters.

Yup, sir got the issue and fixed it. The script was removing only full stop but not any other punctuations. If there's again any issue please tell and we will fix it.
Thanks

I think we should not remove the ।।1.38।। thing
this helps us in full text search to find the verses,