File size: 2,701 Bytes
9235851 b943174 9235851 0b33c64 35cc9cd 0b33c64 e4808e7 0b33c64 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 | ---
license: apache-2.0
widget:
- text: "<|endoftext|>\ndef load_excel(path):\n return pd.read_excel(path)\n# docstring\n\"\"\""
---
## Basic info
model based [Salesforce/codegen-350M-mono](https://huggingface.co/Salesforce/codegen-350M-mono)
fine-tuned with data [codeparrot/github-code-clean](https://huggingface.co/datasets/codeparrot/github-code-clean)
data filter by python
## Usage
```python
from transformers import AutoTokenizer, AutoModelForCausalLM
model_type = 'kdf/python-docstring-generation'
tokenizer = AutoTokenizer.from_pretrained(model_type)
model = AutoModelForCausalLM.from_pretrained(model_type)
inputs = tokenizer('''<|endoftext|>
def load_excel(path):
return pd.read_excel(path)
# docstring
"""''', return_tensors='pt')
doc_max_length = 128
generated_ids = model.generate(
**inputs,
max_length=inputs.input_ids.shape[1] + doc_max_length,
do_sample=False,
return_dict_in_generate=True,
num_return_sequences=1,
output_scores=True,
pad_token_id=50256,
eos_token_id=50256 # <|endoftext|>
)
ret = tokenizer.decode(generated_ids.sequences[0], skip_special_tokens=False)
print(ret)
```
## Prompt
You could give model a style or a specific language, for example:
```python
inputs = tokenizer('''<|endoftext|>
def add(a, b):
return a + b
# docstring
"""
Calculate numbers add.
Args:
a: the first number to add
b: the second number to add
Return:
The result of a + b
"""
<|endoftext|>
def load_excel(path):
return pd.read_excel(path)
# docstring
"""''', return_tensors='pt')
doc_max_length = 128
generated_ids = model.generate(
**inputs,
max_length=inputs.input_ids.shape[1] + doc_max_length,
do_sample=False,
return_dict_in_generate=True,
num_return_sequences=1,
output_scores=True,
pad_token_id=50256,
eos_token_id=50256 # <|endoftext|>
)
ret = tokenizer.decode(generated_ids.sequences[0], skip_special_tokens=False)
print(ret)
inputs = tokenizer('''<|endoftext|>
def add(a, b):
return a + b
# docstring
"""
计算数字相加
Args:
a: 第一个加数
b: 第二个加数
Return:
相加的结果
"""
<|endoftext|>
def load_excel(path):
return pd.read_excel(path)
# docstring
"""''', return_tensors='pt')
doc_max_length = 128
generated_ids = model.generate(
**inputs,
max_length=inputs.input_ids.shape[1] + doc_max_length,
do_sample=False,
return_dict_in_generate=True,
num_return_sequences=1,
output_scores=True,
pad_token_id=50256,
eos_token_id=50256 # <|endoftext|>
)
ret = tokenizer.decode(generated_ids.sequences[0], skip_special_tokens=False)
print(ret)
``` |