From be9fba2d322e5435423570ae6c59cf90a92cf71e Mon Sep 17 00:00:00 2001 From: Domenic Denicola Date: Fri, 7 Mar 2025 13:29:54 +0900 Subject: [PATCH] Incorporate inputQuota, measureInputUsage(), and quota exceeded errors Follows https://github.com/webmachinelearning/writing-assistance-apis/pull/43. --- README.md | 64 ++++++++++++++++++++ index.bs | 177 +++++++++++++++++++++++++++++++++++++++++++++++------- 2 files changed, 220 insertions(+), 21 deletions(-) diff --git a/README.md b/README.md index d008fc6..6c3df87 100644 --- a/README.md +++ b/README.md @@ -175,6 +175,70 @@ It is also nicely future-extensible by adding more events and properties to the Finally, note that there is a sort of precedent in the (never-shipped) [`FetchObserver` design](https://github.com/whatwg/fetch/issues/447#issuecomment-281731850). +### Too-large inputs + +It's possible that the inputs given for translation or language detection might be too large for the underlying machine learning model to handle. Although there are often techniques that allow implementations to break up the inputs into smaller chunks, and combine the results, the APIs have some facilities to allow browsers to signal such too-large inputs. + +Whenever any API call fails due to too-large input, it is rejected with a `QuotaExceededError`. This is a proposed new type of exception, which subclasses `DOMException`, and replaces the web platform's existing `"QuotaExceededError"` `DOMException`. See [whatwg/webidl#1465](https://github.com/whatwg/webidl/pull/1465) for this proposal. For our purposes, the important part is that it has the following properties: + +* `requested`: how much "usage" the input consists of +* `quota`: how much "usage" was available (which will be less than `requested`) + +The "usage" concept is specific to the implementation, and could be something like string length, or [language model tokens](https://arxiv.org/abs/2404.08335). + +This allows detecting failures due to overlarge inputs and giving clear feedback to the user, with code such as the following: + +```js +const detector = await ai.languageDetector.create(); + +try { + console.log(await detector.detect(potentiallyLargeInput)); +} catch (e) { + if (e.name === "QuotaExceededError") { + console.error(`Input too large! You tried to detect the language of ${e.requested} tokens, but ${e.quota} is the max supported.`); + + // Or maybe: + console.error(`Input too large! It's ${e.requested / e.quota}x as large as the maximum possible input size.`); + } +} +``` + +In some cases, instead of providing errors after the fact, the developer needs to be able to communicate to the user how close they are to the limit. For this, they can use the `inputQuota` property and the `measureInputUsage()` method on the translator or language detector objects: + +```js +const translator = await ai.translator.create({ + sourceLanguage: "en", + targetLanguage: "jp" +}); +meterEl.max = translator.inputQuota; + +textbox.addEventListener("input", () => { + meterEl.value = await translator.measureInputUsage(textbox.value); + submitButton.disabled = meterEl.value > meterEl.max; +}); + +submitButton.addEventListener("click", () => { + console.log(translator.translate(textbox.value)); +}); +``` + +Note that if an implementation does not have any limits, e.g. because it uses techniques to split up the input and process it a bit at a time, then `inputQuota` will be `+Infinity` and `measureInputUsage()` will always return 0. + +Developers need to be cautious not to over-use this API, however, as it requires a round-trip to the underlying model. That is, the following code is bad, as it performs two round trips with the same input: + +```js +// DO NOT DO THIS + +const usage = await translator.measureInputUsage(input); +if (usage < translator.inputQuota) { + console.log(await translator.translate(input)); +} else { + console.error(`Input too large!`); +} +``` + +If you're planning to call `translate()` anyway, then using a pattern like the one that opened this section, which catches `QuotaExceededError`s, is more efficient than using `measureInputUsage()` plus a conditional call to `translate()`. + ### Destruction and aborting The API comes equipped with a couple of `signal` options that accept `AbortSignal`s, to allow aborting the creation of the translator/language detector, or the translation/language detection operations themselves: diff --git a/index.bs b/index.bs index 02bbf18..8bb7502 100644 --- a/index.bs +++ b/index.bs @@ -23,6 +23,12 @@ urlPrefix: https://tc39.es/ecma402/; spec: ECMA-402 text: Unicode canonicalized locale identifier; url: sec-language-tags type: abstract-op text: LookupMatchingLocaleByBestFit; url: sec-lookupmatchinglocalebybestfit +urlPrefix: https://whatpr.org/webidl/1465.html; spec: WEBIDL + type: interface + text: QuotaExceededError; url: quotaexceedederror + type: dfn; for: QuotaExceededError + text: requested; url: quotaexceedederror-requested + text: quota; url: quotaexceedederror-quota

Introduction

@@ -55,6 +61,12 @@ interface AITranslator { readonly attribute DOMString sourceLanguage; readonly attribute DOMString targetLanguage; + + Promise measureInputUsage( + DOMString input, + optional AITranslatorTranslateOptions options = {} + ); + readonly attribute unrestricted double inputQuota; }; AITranslator includes AIDestroyable; @@ -124,9 +136,9 @@ The translator getter steps are to return [=this=] This could include loading the model into memory, or loading any fine-tunings necessary to support the specific options in question. - 1. If initialization failed for any reason, then return false. + 1. If initialization failed for any reason, then return a [=DOMException error information=] whose [=DOMException error information/name=] is "{{OperationError}}" and whose [=DOMException error information/details=] contain appropriate detail. - 1. Return true. + 1. Return null.
@@ -134,6 +146,8 @@ The translator getter steps are to return [=this=] 1. [=Assert=]: these steps are running on |realm|'s [=ECMAScript/surrounding agent=]'s [=agent/event loop=]. + 1. Let |inputQuota| be the amount of input quota that is available to the user agent for future [=translate|translation=] operations. (This value is [=implementation-defined=], and may be +∞ if there are no specific limits beyond, e.g., the user's memory, or the limits of JavaScript strings.) + 1. Return a new {{AITranslator}} object, created in |realm|, with
@@ -142,6 +156,9 @@ The translator getter steps are to return [=this=] : [=AITranslator/target language=] :: |options|["{{AITranslatorCreateCoreOptions/targetLanguage}}"] + + : [=AITranslator/input quota=] + :: |inputQuota|
@@ -295,18 +312,22 @@ Every {{AITranslator}} has a source language, a [= Every {{AITranslator}} has a target language, a [=string=], set during creation. +Every {{AITranslator}} has an input quota, a [=number=], set during creation. +
The sourceLanguage getter steps are to return [=this=]'s [=AITranslator/source language=]. The targetLanguage getter steps are to return [=this=]'s [=AITranslator/target language=]. +The inputQuota getter steps are to return [=this=]'s [=AITranslator/input quota=]. +
The translate(|input|, |options|) method steps are: - 1. Let |operation| be an algorithm step which takes arguments |chunkProduced|, |done|, |error|, and |stopProducing|, and [=translates=] |input| given [=this=]'s [=AITranslator/source language=], [=this=]'s [=AITranslator/target language=], |chunkProduced|, |done|, |error|, and |stopProducing|. + 1. Let |operation| be an algorithm step which takes arguments |chunkProduced|, |done|, |error|, and |stopProducing|, and [=translates=] |input| given [=this=]'s [=AITranslator/source language=], [=this=]'s [=AITranslator/target language=], [=this=]'s [=AITranslator/input quota=], |chunkProduced|, |done|, |error|, and |stopProducing|. 1. Return the result of [=getting an aggregated AI model result=] given [=this=], |options|, and |operation|.
@@ -314,11 +335,19 @@ The targetLanguage getter steps are to r
The translateStreaming(|input|, |options|) method steps are: - 1. Let |operation| be an algorithm step which takes arguments |chunkProduced|, |done|, |error|, and |stopProducing|, and [=translates=] |input| given [=this=]'s [=AITranslator/source language=], [=this=]'s [=AITranslator/target language=], |chunkProduced|, |done|, |error|, and |stopProducing|. + 1. Let |operation| be an algorithm step which takes arguments |chunkProduced|, |done|, |error|, and |stopProducing|, and [=translates=] |input| given [=this=]'s [=AITranslator/source language=], [=this=]'s [=AITranslator/target language=], [=this=]'s [=AITranslator/input quota=], |chunkProduced|, |done|, |error|, and |stopProducing|. 1. Return the result of [=getting a streaming AI model result=] given [=this=], |options|, and |operation|.
+
+ The measureInputUsage(|input|, |options|) method steps are: + + 1. Let |measureUsage| be an algorithm step which takes argument |stopMeasuring|, and returns the result of [=measuring translator input usage=] given |input|, [=this=]'s [=AITranslator/source language=], [=this=]'s [=AITranslator/target language=], and |stopMeasuring|. + + 1. Return the result of [=measuring AI model input usage=] given [=this=], |options|, and |measureUsage|. +
+

Translation

The algorithm

@@ -329,6 +358,7 @@ The targetLanguage getter steps are to r * a [=string=] |input|, * a [=Unicode canonicalized locale identifier=] |sourceLanguage|, * a [=Unicode canonicalized locale identifier=] |targetLanguage|, + * a [=number=] |inputQuota|, * an algorithm |chunkProduced| that takes a string and returns nothing, * an algorithm |done| that takes no arguments and returns nothing, * an algorithm |error| that takes [=error information=] and returns nothing, and @@ -338,6 +368,28 @@ The targetLanguage getter steps are to r 1. [=Assert=]: this algorithm is running [=in parallel=]. + 1. Let |requested| be the result of [=measuring translator input usage=] given |input|, |sourceLanguage|, |targetLanguage|, and |stopProducing|. + + 1. If |requested| is null, then return. + + 1. If |requested| is an [=error information=], then: + + 1. Perform |error| given |requested|. + + 1. Return. + + 1. [=Assert=]: |requested| is a number. + + 1. If |requested| is greater than |inputQuota|, then: + + 1. Let |errorInfo| be a [=quota exceeded error information=] with a [=quota exceeded error information/requested=] of |requested| and a [=quota exceeded error information/quota=] of |inputQuota|. + + 1. Perform |error| given |errorInfo|. + + 1. Return. + +

In reality, we expect that implementations will check the input usage against the quota as part of the same call into the model as the translation itself. The steps are only separated in the specification for ease of understanding. + 1. In an [=implementation-defined=] manner, subject to the following guidelines, begin the processs of translating |input| from |sourceLanguage| into |targetLanguage|. If |input| is the empty string, or otherwise consists of no translatable content (e.g., only contains whitespace, or control characters), then the resulting translation should be |input|. In such cases, |sourceLanguage| and |targetLanguage| should be ignored. @@ -364,13 +416,46 @@ The targetLanguage getter steps are to r 1. Otherwise, if an error occurred during translation: - 1. Let the error be represented as [=error information=] |errorInfo| according to the guidance in [[#translator-errors]]. + 1. Let the error be represented as a [=DOMException error information=] |errorInfo| according to the guidance in [[#translator-errors]]. 1. Perform |error| given |errorInfo|. 1. [=iteration/Break=]. +

Usage

+ +
+ To measure translator input usage, given: + + * a [=string=] |input|, + * a [=Unicode canonicalized locale identifier=] |sourceLanguage|, + * a [=Unicode canonicalized locale identifier=] |targetLanguage|, and + * an algorithm |stopMeasuring| that takes no arguments and returns a boolean, + + perform the following steps: + + 1. [=Assert=]: this algorithm is running [=in parallel=]. + + 1. Let |inputToModel| be the [=implementation-defined=] string that would be sent to the underlying model in order to [=translate=] |input| from |sourceLanguage| to |targetLanguage|. + +

This might be just |input| itself, if |sourceLanguage| and |targetLanguage| were loaded into the model during initialization. Or it might consist of more, e.g. appropriate quota usage for encoding the languages in question, or some sort of wrapper prompt to a language model. + + If during this process |stopMeasuring| starts returning true, then return null. + + If an error occurs during this process, then return an appropriate [=DOMException error information=] according to the guidance in [[#translator-errors]]. + + 1. Return the amount of input usage needed to represent |inputToModel| when given to the underlying model. The exact calculation procedure is [=implementation-defined=], subject to the following constraints. + + The returned input usage must be nonnegative and finite. It must be 0, if there are no usage quotas for the translation process (i.e., if the [=AITranslator/input quota=] is +∞). Otherwise, it must be positive and should be roughly proportional to the [=string/length=] of |inputToModel|. + +

This might be the number of tokens needed to represent |input| in a language model tokenization scheme, or it might be |input|'s [=string/length=]. It could also be some variation of these which also counts the usage of any prefixes or suffixes necessary to give to the model. + + If during this process |stopMeasuring| starts returning true, then instead return null. + + If an error occurs during this process, then instead return an appropriate [=DOMException error information=] according to the guidance in [[#translator-errors]]. +

+

Errors

When translation fails, the following possible reasons may be surfaced to the web developer. This table lists the possible {{DOMException}} [=DOMException/names=] and the cases in which an implementation should use them: @@ -389,17 +474,13 @@ When translation fails, the following possible reasons may be surfaced to the we "{{NotReadableError}}"

The translation output was filtered by the user agent, e.g., because it was detected to be harmful, inaccurate, or nonsensical. - - "{{QuotaExceededError}}" - -

The input to be translated was too large for the user agent to handle. "{{UnknownError}}"

All other scenarios, or if the user agent would prefer not to disclose the failure reason. -

This table does not give the complete list of exceptions that can be surfaced by {{AITranslator/translate()|translator.translate()}} and {{AITranslator/translateStreaming()|translator.translateStreaming()}}. It only contains those which can come from the [=implementation-defined=] [=translate=] algorithm. +

This table does not give the complete list of exceptions that can be surfaced by the translator API. It only contains those which can come from certain [=implementation-defined=] steps.

The language detector API

@@ -427,8 +508,13 @@ interface AILanguageDetector { readonly attribute FrozenArray? expectedInputLanguages; - undefined destroy(); + Promise measureInputUsage( + DOMString input, + optional AITranslatorTranslateOptions options = {} + ); + readonly attribute unrestricted double inputQuota; }; +AILanguageDetector includes AIDestroyable; dictionary AILanguageDetectorCreateCoreOptions { sequence expectedInputLanguages; @@ -498,9 +584,9 @@ The languageDetector getter steps are to return [= This could include loading the model into memory, or loading any fine-tunings necessary to support the languages identified in |options|["{{AILanguageDetectorCreateCoreOptions/expectedInputLanguages}}"]. - 1. If initialization failed for any reason, then return false. + 1. If initialization failed for any reason, then return a [=DOMException error information=] whose [=DOMException error information/name=] is "{{OperationError}}" and whose [=DOMException error information/details=] contain appropriate detail. - 1. Return true. + 1. Return null.
@@ -508,11 +594,16 @@ The languageDetector getter steps are to return [= 1. [=Assert=]: these steps are running on |realm|'s [=ECMAScript/surrounding agent=]'s [=agent/event loop=]. + 1. Let |inputQuota| be the amount of input quota that is available to the user agent for future [=detect languages|language detection=] operations. (This value is [=implementation-defined=], and may be +∞ if there are no specific limits beyond, e.g., the user's memory, or the limits of JavaScript strings.) + 1. Return a new {{AILanguageDetector}} object, created in |realm|, with
: [=AILanguageDetector/expected input languages=] :: the result of [=creating a frozen array=] given |options|["{{AILanguageDetectorCreateCoreOptions/expectedInputLanguages}}"] if it [=set/is empty|is not empty=]; otherwise null + + : [=AILanguageDetector/input quota=] + :: |inputQuota|
@@ -576,10 +667,14 @@ The languageDetector getter steps are to return [= Every {{AILanguageDetector}} has an expected input languages, a {{FrozenArray}}<{{DOMString}}> or null, set during creation. +Every {{AILanguageDetector}} has an input quota, a [=number=], set during creation. +
The expectedInputLanguages getter steps are to return [=this=]'s [=AILanguageDetector/expected input languages=]. +The inputQuota getter steps are to return [=this=]'s [=AILanguageDetector/input quota=]. +
@@ -612,13 +707,13 @@ The expectedInputLanguages getter 1. Return |abortedDuringOperation|. - 1. Let |result| be the result of [=detecting languages=] given |input| and |stopProducing|. + 1. Let |result| be the result of [=detecting languages=] given |input|, [=this=]'s [=AILanguageDetector/input quota=], and |stopProducing|. 1. [=Queue a global task=] on the [=AI task source=] given [=this=]'s [=relevant global object=] to perform the following steps: 1. If |abortedDuringOperation| is true, then [=reject=] |promise| with |compositeSignal|'s [=AbortSignal/abort reason=]. - 1. Otherwise, if |result| is an [=error information=], then [=reject=] |promise| with the result of [=exception/creating=] a {{DOMException}} with name given by |errorInfo|'s [=error information/error name=], using |errorInfo|'s [=error information/error information=] to populate the message appropriately. + 1. Otherwise, if |result| is an [=error information=], then [=reject=] |promise| with the result of [=converting error information into an exception object=] given |result|. 1. Otherwise: @@ -627,13 +722,31 @@ The expectedInputLanguages getter 1. [=Resolve=] |promise| with |result|. +
+ The measureInputUsage(|input|, |options|) method steps are: + + 1. Let |measureUsage| be an algorithm step which takes argument |stopMeasuring|, and returns the result of [=measuring language detector input usage=] given |input| and |stopMeasuring|. + + 1. Return the result of [=measuring AI model input usage=] given [=this=], |options|, and |measureUsage|. +
+

The algorithm

- To detect languages given a [=string=] |input| and an algorithm |stopProducing| that takes no arguments and returns a boolean, perform the following steps. They will return either null, an [=error information=], or a [=list=] of {{LanguageDetectionResult}} dictionaries. + To detect languages given a [=string=] |input|, a [=number=] |inputQuota|, and an algorithm |stopProducing| that takes no arguments and returns a boolean, perform the following steps. They will return either null, an [=error information=], or a [=list=] of {{LanguageDetectionResult}} dictionaries. 1. [=Assert=]: this algorithm is running [=in parallel=]. + 1. Let |requested| be the result of [=measuring language detector input usage=] given |input| and |stopProducing|. + + 1. If |requested| is null or an [=error information=], then return |requested|. + + 1. [=Assert=]: |requested| is a number. + + 1. If |requested| is greater than |inputQuota|, then return a [=quota exceeded error information=] with a [=quota exceeded error information/requested=] of |requested| and a [=quota exceeded error information/quota=] of |inputQuota|. + +

In reality, we expect that implementations will check the input usage against the quota as part of the same call into the model as the language detection itself. The steps are only separated in the specification for ease of understanding. + 1. Let |availabilities| be the result of [=getting language availabilities=] given the purpose of detecting text written in that language. 1. Let |currentlyAvailableLanguages| be |availabilities|["{{AIAvailability/available}}"]. @@ -687,6 +800,32 @@ The expectedInputLanguages getter

The post-processing of |rawResult| and |unknown| essentially consolidates all languages below a certain threshold into the "`und`" language. Languages which are less than 1% likely, or contribute to less than 1% of the text, are considered more likely to be noise than to be worth detecting. Similarly, if the implementation is less sure about a language than it is about the text not being in any of the languages it knows, that language is probably not worth returning to the web developer.

+

Usage

+ +
+ To measure language detector input usage, given a [=string=] |input| and an algorithm |stopMeasuring| that takes no arguments and returns a boolean, perform the following steps: + + 1. [=Assert=]: this algorithm is running [=in parallel=]. + + 1. Let |inputToModel| be the [=implementation-defined=] string that would be sent to the underlying model in order to [=detect languages=] given |input|. + +

This might be just |input| itself, or it might include some sort of wrapper prompt to a language model. + + If during this process |stopMeasuring| starts returning true, then return null. + + If an error occurs during this process, then return an appropriate [=DOMException error information=] according to the guidance in [[#language-detector-errors]]. + + 1. Return the amount of input usage needed to represent |inputToModel| when given to the underlying model. The exact calculation procedure is [=implementation-defined=], subject to the following constraints. + + The returned input usage must be nonnegative and finite. It must be 0, if there are no usage quotas for the translation process (i.e., if the [=AILanguageDetector/input quota=] is +∞). Otherwise, it must be positive and should be roughly proportional to the [=string/length=] of |inputToModel|. + +

This might be the number of tokens needed to represent |input| in a language model tokenization scheme, or it might be |input|'s [=string/length=]. It could also be some variation of these which also counts the usage of any prefixes or suffixes necessary to give to the model. + + If during this process |stopMeasuring| starts returning true, then instead return null. + + If an error occurs during this process, then instead return an appropriate [=DOMException error information=] according to the guidance in [[#language-detector-errors]]. +

+

Errors

When language detection fails, the following possible reasons may be surfaced to the web developer. This table lists the possible {{DOMException}} [=DOMException/names=] and the cases in which an implementation should use them: @@ -701,14 +840,10 @@ When language detection fails, the following possible reasons may be surfaced to "{{NotAllowedError}}"

Language detection is disabled by user choice or user agent policy. - - "{{QuotaExceededError}}" - -

The input to be detected was too large for the user agent to handle. "{{UnknownError}}"

All other scenarios, or if the user agent would prefer not to disclose the failure reason. -

This table does not give the complete list of exceptions that can be surfaced by {{AILanguageDetector/detect()|detector.detect()}}. It only contains those which can come from the [=implementation-defined=] [=detect languages=] algorithm. +

This table does not give the complete list of exceptions that can be surfaced by the language detector API. It only contains those which can come from certain [=implementation-defined=] steps.