codeparrot/github-jupyter-code-to-text

Dataset description

This dataset consists of sequences of Python code followed by a a docstring explaining its function. It was constructed by concatenating code and text pairs from this dataset that were originally code and markdown cells in Jupyter Notebooks.

The content of each example the following:

[CODE]
"""
Explanation: [TEXT]
End of explanation
"""
[CODE]
"""
Explanation: [TEXT]
End of explanation
"""
...

How to use it

from datasets import load_dataset

ds = load_dataset("codeparrot/github-jupyter-code-to-text", split="train")

Dataset({
    features: ['repo_name', 'path', 'license', 'content'],
    num_rows: 47452
})