Skip to content

Commit

Permalink
Update AI Provider plugin documentation for provider instances.
Browse files Browse the repository at this point in the history 
See: MDL-82977
  • Loading branch information
Matt Porritt committed Jan 3, 2025
1 parent d68ae56 commit eefc72a
Showing 1 changed file with 139 additions and 101 deletions.
240 changes: 139 additions & 101 deletions docs/apis/plugintypes/ai/provider.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,9 @@ Incoming data to the Provider plugin arrives via the Manager `core_ai\manager`.
The Manager is the connective tissue between the Provider and the [Placement](/apis/plugintypes/ai/placement.md) plugins.
Likewise, all responses from the Provider plugin are handed back to the Manager before being passed to the Placement plugin.

A Provider plugin allows many "provider instances" to be defined, each of which can support a different set of configurations.
This facilitates having providers for specific tasks. So, you can use a more efficient model for lightweight tasks like summarisation, and a more fully featured model for text generation. Another example is defaulting to using a cheaper model with a lower token limit, and then falling back to a more expensive model if a request is too large for the default model.

:::warning The Golden Rule:

Placements **do not** know about Providers, and Providers **do not** know about Placements.
Expand All @@ -28,14 +31,17 @@ The naming convention for a Provider class is `aiprovider_<plugin name>`.
For example: `aiprovider_openai`, or `aiprovider_azureai` (with a corresponding namespace).

Each Provider **must** inherit from the `\core_ai\provider` abstract class.

### Required Methods

They must also implement the following methods:

**`get_action_list(): array`**

This is the list of Actions that are supported by this Provider, for example the `aiprovider_openai` plugin defines this as:

```php
public function get_action_list(): array {
public static function get_action_list(): array {
return [
\core_ai\aiactions\generate_text::class,
\core_ai\aiactions\generate_image::class,
Expand All @@ -57,7 +63,7 @@ the result.

```php
public function is_provider_configured(): bool {
return !empty($this->apikey) && !empty($this->apiendpoint);
return !empty($this->config['apikey']) && !empty($this->config['endpoint']);
}
```

Expand Down Expand Up @@ -108,18 +114,34 @@ files the developer is going to use.

```console
.
├── amd
│   ├── build
│   │   ├── modelchooser.min.js
│   │   └── modelchooser.min.js.map
│   └── src
│   └── modelchooser.js
├── classes
│   ├── abstract_processor.php
│   ├── aimodel
│   │   ├── gpt4o.php
│   │   └── o1.php
│   ├── form
│   │   ├── action_form.php
│   │   ├── action_generate_image_form.php
│   │   └── action_generate_text_form.php
│   ├── helper.php
│   ├── hook_listener.php
│   ├── privacy
│   │   └── provider.php
│   ├── process_generate_image.php
│   ├── process_generate_text.php
│   ├── process_summarise_text.php
│   └── provider.php
├── db
│   └── hooks.php
├── lang
│   └── en
│   └── aiprovider_openai.php
├── settings.php
├── tests
│   ├── fixtures
│   │   ├── image_request_success.json
Expand All @@ -135,121 +157,137 @@ files the developer is going to use.

</details>

## Settings
## Provider Settings

Settings for the Provider are defined as a [Hook](/apis/core/hooks/index.md).
Each Provider plugin should create a new `classes/hook_listener.php` file. This file should contain a class with a static method that defines the hook callback.
create a new admin settings page using `core_ai\admin\admin_settingspage_provider` class. This class should implement a `set_form_definition_for_aiprovider_<plugin name>` method.

Settings for the Provider should be defined in the `settings.php` file.
Each Provider plugin should create a new admin settings page using `core_ai\admin\admin_settingspage_provider` class.
This method should define the settings form for the provider plugin, using the provided `mform` object.

For example, the `aiprovider_openai` plugin defines this:

```php
use core_ai\admin\admin_settingspage_provider;

if ($hassiteconfig) {
// Provider specific settings heading.
$settings = new admin_settingspage_provider(
'aiprovider_openai',
new lang_string('pluginname', 'aiprovider_openai'),
'moodle/site:config',
true,
);
...
```
namespace aiprovider_openai;
use core_ai\hook\after_ai_provider_form_hook;

class hook_listener {

/**
* Hook listener for the Open AI instance setup form.
*
* @param after_ai_provider_form_hook $hook The hook to add to the AI instance setup.
*/
public static function set_form_definition_for_aiprovider_openai(after_ai_provider_form_hook $hook): void {
if ($hook->plugin !== 'aiprovider_openai') {
return;
}

## Rate limiting
$mform = $hook->mform;

// Required setting to store OpenAI API key.
$mform->addElement(
'passwordunmask',
'apikey',
get_string('apikey', 'aiprovider_openai'),
['size' => 75],
);
$mform->addHelpButton('apikey', 'apikey', 'aiprovider_openai');
$mform->addRule('apikey', get_string('required'), 'required', null, 'client');

// Setting to store OpenAI organization ID.
$mform->addElement(
'text',
'orgid',
get_string('orgid', 'aiprovider_openai'),
['size' => 25],
);
$mform->setType('orgid', PARAM_TEXT);
$mform->addHelpButton('orgid', 'orgid', 'aiprovider_openai');

It is recommended that Providers implement rate limiting to prevent abuse of the external AI services.
}

}
```

To assist with this, the AI subsystem provides a `core_ai\rate_limiter` class that can be used to implement rate limiting.
This class supports both user and system level rate limiting.
Because a hook is used to define the settings, Provider plugins also need to have a `db/hooks.php` file to register its hook callback.
The specified plugin callback method is called whenever the provider instance is chosen in the provider administration settings.

This should be implemented in a `is_request_allowed()` method in the Provider class. For example, from the
`aiprovider_openai` plugin:
For example, the `aiprovider_openai` plugin defines this:

```php
/**
* Check if the request is allowed by the rate limiter.
*
* @param aiactions\base $action The action to check.
* @return array|bool True on success, array of error details on failure.
*/
public function is_request_allowed(aiactions\base $action): array|bool {
$ratelimiter = \core\di::get(rate_limiter::class);
$component = \core\component::get_component_from_classname(get_class($this));

// Check the user rate limit.
if ($this->enableuserratelimit) {
if (!$ratelimiter->check_user_rate_limit(
component: $component,
ratelimit: $this->userratelimit,
userid: $action->get_configuration('userid')
)) {
return [
'success' => false,
'errorcode' => 429,
'errormessage' => 'User rate limit exceeded',
];
}
}
defined('MOODLE_INTERNAL') || die();

$callbacks = [
[
'hook' => \core_ai\hook\after_ai_provider_form_hook::class,
'callback' => \aiprovider_openai\hook_listener::class . '::set_form_definition_for_aiprovider_openai',
],
];
```

// Check the global rate limit.
if ($this->enableglobalratelimit) {
if (!$ratelimiter->check_global_rate_limit(
component: $component,
ratelimit: $this->globalratelimit
)) {
return [
'success' => false,
'errorcode' => 429,
'errormessage' => 'Global rate limit exceeded',
];
}
}
## Action Settings

Each of the actions that a provider plugin supports can have its own settings.
If an action requires additional settings, the provider class for the plugin should override the `get_action_settings()` method.
The method must return an instance of `core_ai\form\action_settings_form`.

For example, the `aiprovider_openai` plugin defines this:

return true;
```php
#[\Override]
public static function get_action_settings(
string $action,
array $customdata = [],
): action_settings_form|bool {
$actionname = substr($action, (strrpos($action, '\\') + 1));
$customdata['actionname'] = $actionname;
$customdata['action'] = $action;
if ($actionname === 'generate_text' || $actionname === 'summarise_text') {
return new form\action_generate_text_form(customdata: $customdata);
} else if ($actionname === 'generate_image') {
return new form\action_generate_image_form(customdata: $customdata);
}

return false;
}
```

If implementing rate limiting, settings for the rate limits should be provided in the plugin settings.
The actual settings form for the actions should be defined in the `classes/form` directory.
For example, the `aiprovider_openai` plugin defines `action_generate_text_form` and `action_generate_image_form` classes.
These classes should extend the `core_ai\form\action_settings_form` class, and must implement the `definition()` method.

For example, the `aiprovider_openai` plugin provides settings for the user and global rate limits:
For example, the `aiprovider_openai` plugin defines this:

```php
// Setting to enable/disable global rate limiting.
$settings->add(new admin_setting_configcheckbox(
'aiprovider_openai/enableglobalratelimit',
new lang_string('enableglobalratelimit', 'aiprovider_openai'),
new lang_string('enableglobalratelimit_desc', 'aiprovider_openai'),
0,
));

// Setting to set how many requests per hour are allowed for the global rate limit.
// Should only be enabled when global rate limiting is enabled.
$settings->add(new admin_setting_configtext(
'aiprovider_openai/globalratelimit',
new lang_string('globalratelimit', 'aiprovider_openai'),
new lang_string('globalratelimit_desc', 'aiprovider_openai'),
100,
PARAM_INT,
));
$settings->hide_if('aiprovider_openai/globalratelimit', 'aiprovider_openai/enableglobalratelimit', 'eq', 0);

// Setting to enable/disable user rate limiting.
$settings->add(new admin_setting_configcheckbox(
'aiprovider_openai/enableuserratelimit',
new lang_string('enableuserratelimit', 'aiprovider_openai'),
new lang_string('enableuserratelimit_desc', 'aiprovider_openai'),
0,
));

// Setting to set how many requests per hour are allowed for the user rate limit.
// Should only be enabled when user rate limiting is enabled.
$settings->add(new admin_setting_configtext(
'aiprovider_openai/userratelimit',
new lang_string('userratelimit', 'aiprovider_openai'),
new lang_string('userratelimit_desc', 'aiprovider_openai'),
10,
PARAM_INT,
));
$settings->hide_if('aiprovider_openai/userratelimit', 'aiprovider_openai/enableuserratelimit', 'eq', 0);
class action_generate_text_form extends action_settings_form {
#[\Override]
protected function definition() {
$mform = $this->_form;
$actionconfig = $this->_customdata['actionconfig']['settings'] ?? [];
$returnurl = $this->_customdata['returnurl'] ?? null;
$actionname = $this->_customdata['actionname'];
$action = $this->_customdata['action'];
$providerid = $this->_customdata['providerid'] ?? 0;

// Action model to use.
$mform->addElement(
'text',
'model',
get_string("action:{$actionname}:model", 'aiprovider_openai'),
'maxlength="255" size="20"',
);
$mform->setType('model', PARAM_TEXT);
$mform->addRule('model', null, 'required', null, 'client');
$mform->setDefault('model', $actionconfig['model'] ?? 'gpt-4o');
$mform->addHelpButton('model', "action:{$actionname}:model", 'aiprovider_openai');

...
```

## Rate limiting

Provider plugins by default implement rate limiting to prevent abuse of the external AI services.
This is inherited from the `core_ai\provider` class. Developers don't need to implement rate limiting themselves.

This default rate limiting behaviour can be changed by overriding the `is_request_allowed()` method `core_ai\provider` class.

0 comments on commit eefc72a

Please sign in to comment.