llama-2-70b

This is LLama v2, the latest language model from Meta. Compared to LLaMA v1, it was twRAIned on more data (~2 trillion tokens) and supports context windows up to 4k tokens. This endpoint uses the 70 billion parameter model, which is the largest LLama v2 model released by Meta so far.

Hold: 0

Required: 25,000 $wRAI

51.3k runs

Input

prompt

Prompt to send to Llama v2.

system_prompt

System prompt to send to Llama v2. This is prepended to the prompt and helps guide system behavior.

max_new_tokens

Maximum number of tokens to generate. A word is generally 2-3 tokens (minimum: 1)

min_new_tokens

Minimum number of tokens to generate. To disable, set to -1. A word is generally 2-3 tokens. (minimum: -1)

temperature

Adjusts randomness of outputs, greater than 1 is random and 0 is deterministic, 0.75 is a good starting value. (minimum: 0.01; maximum: 5)

top_p

When decoding text, samples from the top p percentage of most likely tokens; lower to ignore less likely tokens (maximum: 1)

top_k

When decoding text, samples from the top k most likely tokens; lower to ignore less likely tokens

repetition_penalty

Penalty for repeated words in generated text; 1 is no penalty, values greater than 1 discourage repetition, less than 1 encourage it. (minimum: 0.01; maximum: 5)

repetition_penalty_sustain

Number of most recent tokens to apply repetition penalty to, -1 to apply to whole context (minimum: -1)

token_repetition_penalty_decay

Gradually decrease penalty over this many tokens (minimum: 1)

Hold at least 25,000 wRAI to use this model

Multi AI platform is completely free, but most models are only accessible to wRAI token holders. If you have any questions, feel free to ask in our Telegram chat

Output

Download

Input

max_new_tokens

500

min_new_tokens

prompt

Can you write a poem about open source machine learning? Let's make it in the style of E. E. Cummings.

repetition_penalty

repetition_penalty_sustain

256

system_prompt

You are a helpful, respectful and honest assistant. Always answer as helpfully as possible, while being safe. Your answers should not include any harmful, unethical, racist, sexist, toxic, dangerous, or illegal content. Please ensure that your responses are socially unbiased and positive in nature. If a question does not make any sense, or is not factually coherent, explain why instead of answering something not correct. If you don't know the answer to a question, please don't share false information.

temperature

0.5

token_repetition_penalty_decay

128

top_k

250

top_p

Output

Download