Avoid loading full datasets when generating benchmark dataset details

[R-Palazzo]

Resolve #609
86ba43c4v

Tested for the multi-table benchmark upload workflow here

[codecov]

Codecov Report

❌ Patch coverage is 94.33962% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 85.83%. Comparing base (30ef01a) to head (698455e).

Files with missing lines	Patch %	Lines
sdgym/dataset_explorer.py	95.23%	2 Missing ⚠️
sdgym/s3.py	90.90%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #622      +/-   ##
==========================================
+ Coverage   85.81%   85.83%   +0.02%     
==========================================
  Files          40       40              
  Lines        3722     3771      +49     
==========================================
+ Hits         3194     3237      +43     
- Misses        528      534       +6     

Flag	Coverage Δ
integration	44.41% <84.90%> (+0.24%)	⬆️
unit	81.75% <94.33%> (+0.15%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[sarahmish]

Do we want to calculate the number of cells in the dataset too? number of rows x number of columns?

[sarahmish]

I think this can be optimized

Suggested change

	count = 0
	has_header = False
	for row in reader:
	if not row:
	continue
	if not has_header:
	has_header = True
	continue
	count += 1
	next(reader) # skip header
	count = sum(1 for row in reader if row)

[sarahmish]

I think you can also count the number of new lines without a csv reader which would make it even faster

def _count_csv_rows(csv_file):
    text_file = io.TextIOWrapper(csv_file, encoding='utf-8-sig', newline='')
    count = sum(1 for line in text_file if line.strip()) - 1 # -1 for the header
    text_file.detach()
    return max(count, 0)

[R-Palazzo]

I kept the csv reader otherwise the tests were failing:
https://github.com/sdv-dev/SDGym/actions/runs/27419559432/job/81041188188

For example, the following was treated as 2 lines

id,text
1,"first line
second line"

[R-Palazzo]

Do we want to calculate the number of cells in the dataset too? number of rows x number of columns?

@sarahmish this is something we could add, maybe in a separate issue. For now in the Dataset_Details table there are the Total_Num_Columns and Total_Num_Rows