Skip to content

Conversation

@davebiagioni
Copy link

Purpose: Allow custom tasks to load datasets from local files, not only from the Hugging Face Hub. Useful for offline / air‑gapped / otherwise restricted environments

Changes

  • Config: Add optional hf_data_files to LightevalTaskConfig.
  • Loader: Forward hf_data_files as data_files to datasets.load_dataset in LightevalTask.download_dataset_worker.
  • Examples: Add examples/custom_tasks_templates/custom_yourbench_task_from_files.py.
  • Docs: Update docs/source/adding-a-custom-task.mdx with file-based usage.

Checklist

  • Tests pass locally
  • Pre-commit hooks pass locally
  • Added/updated documentation

@HuggingFaceDocBuilderDev
Copy link
Collaborator

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Copy link
Member

@NathanHB NathanHB left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking great ! few nits and good to merge if tests pass :)

@davebiagioni
Copy link
Author

@NathanHB review comments addressed. thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants