Skip to main content

Anyscale

Anyscale is a Canadian multinational technology firm that concentrates on enterprise artificial intelligence, specializing particularly in large-scale language models.

Deploying your model​

Anyscale offers the capability to deploy various Chat Models(LLM) under its umbrella.

LLM Model​

Refer to their documentation to understand the latest offerings, with feature and cost comparisons.

Usage with MAAP​

To use Anyscale model with MAAP framework, you would need to feed below values.

  • Config File :​

    Provided below are the values required to be added in config.yaml file in embedding section.

    llms:
        class_name: Anyscale
        model_name: <check_references_below>
        max_tokens: <integer_value>
        temperature: <integer_value>

  • Environment Variable :​

    Below value(s) are to be added in .env file, present at builder/partnerproduct/.

    ANYSCALE_API_KEY = <check_references_below>
    ANYSCALE_BASE_URL= <check_references_below>

References​

Provided below are the instructions on how to procure the right values for building your MAAP framework.

  • Model Name​

    You can pick any model from the updated list given in Anyscale documentation.

  • API Key & Base URL​

    Follow the steps given in this guide, to setup your account and get API Key and Base URL