Text Generation

Job request to generate text using AI models.


Clients MIGHT provide a seed sentence or a prompt in the "i" tag field field for the AI to continue generating from.



Description: Specifies the identifier for the AI model to be used for the text generation task. Different models might have different capabilities, sizes, training data, or fine-tuning, which can lead to variations in output quality and style.

Usage: Set this parameter to the specific ID or name of the AI model you wish to use. This is especially useful if you have multiple models available and you want to select a particular one for a task. Ensure that the specified model is compatible with the task at hand and has been adequately trained or fine-tuned for optimal results.

For instance, if you have a model specifically trained for medical text generation and another for general-purpose tasks, you might use the Model-Id parameter to specify which one to use based on the context of your request.


Description: Defines the maximum number of tokens (words, punctuation marks, etc.) that the generated output should contain. It helps to limit the length of the response and ensures that the output does not exceed a certain size.

Usage: If you want a short response, set a lower value. For longer responses, increase the value.


Description: Controls the randomness of the AI’s output. A higher value makes the output more random, while a lower value makes it more deterministic and focused on the most likely completion.

Usage: Values typically range from 0.1 to 2.0. A value of 1.0 is neutral, below 1.0 makes the model more conservative, and above 1.0 makes it more creative.


Description: Limits the AI to consider only the top K probable next words/tokens when generating a response. It narrows down the choices and can make the output more coherent, especially when set to a reasonable number.

Usage: Common values might range from 20 to 50, but it can vary based on the specific task.


Description: Also known as “nucleus sampling.” Instead of just taking the top K probable words, it selects the smallest set of words whose cumulative probability exceeds a threshold P. This allows for more diversity than Top-K alone.

Usage: Values are typically between 0.7 and 0.95. A higher value results in more randomness in the output.


Description: Applies a penalty to words/tokens that the model has already used, discouraging it from repeating the same words or phrases. This helps to make the output more diverse and reduces redundancy.

Usage: Values greater than 1.0 discourage repetition, while values less than 1.0 encourage it. For instance, setting it to 1.2 might mildly discourage repetition.

NOTE: All parameters are optional and additional application specific parameters can be added.

[ "param", "model", "LLaMA-2" ],
[ "param", "max_tokens", "512" ],
[ "param", "temperature", "0.5" ],
[ "param", "top-k", "50" ],
[ "param", "top-p", "0.7" ],
[ "param", "frequency_penalty", "1" ]


Including but not limited to:

  • text/plain
  • text/markdown


Generates the output based on the input prompt and params

    "content": "",
    "kind": 5050,
    "tags": [
        [ "i", "what is the capital of France? ", "prompt" ],
        [ "param", "model", "LLaMA-2"],
        [ "param",  "max_tokens", "512"],
        [ "param", "temperature", "0.5"],
        [ "param", "top-k", "50"],
        [ "param", "top-p", "0.7"],
        [ "param",  "frequency_penalty", "1"]