Reducing AI cost and context
Limit the tokens used for extraction to reduce cost
By default, extractions in FetchFox look at the more or less the full HTML of a page. The full HTML contains all the possible data needed for extraction, but it is almost always overkill. The full HTML includes lots of extra data like headers, footers, navigation, and irrelevant attributes. Sending all that extra data inflates the number of tokens the AI has to look at, and makes extraction very expensive.
FetchFox offers different extraction modes which reduce the amount of tokens sent to the AI. These modes can be specified using the extract_context
parameter.
Below are the allowed values for extract_context
and their behavior.
text_only
context send only text from the page to the AI.slim_html
context sends a subset of the page HTML to the AI. This subset keeps only a few tags and attributes that often contain valuable data, like<img src="..." />
and<a href="...">
tags, and converts the rest to text.reduce
context use AI to learn how to reduce the context. It takes the full HTML of the page along with the template, and asks the AI to write code that reduces the context. The AI looks at the page structure and template, and figures out what to data to keeep. The code from this process is re-used for subsequent extractions. As such, it adds a one time cost at the start of the process, but context for all extractions is reduced.
As an example, lets look at the cost for an extraction that runs on the full HTML.
The response to this call will be something like this:
Notice the high number of tokens sent to the AI, and corresponding high cost.
Lets run the same request using the text_only
context.
The response to this call will be something like this:
The number of tokens, and therefore cost, is almost 10x lower. Also, the runtime is lower using the text_only
context.