Overview:
"Learn From Content" is a Lexi feature that enables a user to provide a reference document, or point to a URL, and retrieve content for application to a Topic Model.
In its current form, LFC acts as a means of inserting content to a given Topic Model, stored as a Learned Source. A single Topic Model may feature multiple Learned Sources.
Learned Sources are not, at present, an asset distinct from a Topic Model, but a component of a Topic Model itself.
Lexi's revamped Learn From Content feature is now powered by OpenAI LLMs, including but not limited to GPT-4, GPT-4o, and GPT 4-Mini ( o1 integration is not scoped, as of yet).
Content provided to the LFC feature is reviewed for terms relevant to a topic model – proper nouns, jargon, esoteric vocabulary (specific prompts outlined below).
Users may engage via standard or custom prompts.
Custom prompts are assessed for relevance to the process and will cause the process to fail if deemed irrelevant or harmful. e.g. 'Tell me the weather in Copenhagen' , or attempts to spam and overload the Custom Prompt feature with requests, will prompt failure and an error message.
Content uploaded to the Learn from Content feature will not be used by OpenAI in training LLMs.
Technical:
User Permissions: Learning Sources is gated by a Lexi Live permission set, distinct from Topic Models.
View: Load Tab
Edit: Equivalent to View.
Add, Delete: Learn From URL, Learn From File, Manage Entries.
How to: 1. Topic Models: Create or open a Lexi Topic Model and select 'Learn From Content'.
2. On the Learn From Content page, supply a document (Learn from a File) or reference URL (Learn from a Website), featuring the terms you wish to include as Learned Words. The Learn From Content feature will review the input provided and extract key terms – proper nouns, jargon, esoteric vocabulary.
a) Learn From File: - If selecting Learn From a File: supported file types are:
- Word (DOCX, DOC)
- Excel (CSV, XLSX)
- Text (.txt)
b) Learn From Website: - If selecting Learn from a Website, select a URL as specifically related to your content as possible. e.g. link to a specific article rather than a homepage, cnn.com/business rather than cnn.com.
Some sites will block webscraping, in which case the user will receive a corresponding error message.
3. Prompts: Default Prompt: The following prompt applies to all content:
Objective: To generate a list of topic model words based on the input text, which can help improve speech-to-text engines by increasing their recognition accuracy for uncommon words, technical terms, or proper nouns.
- Role:- You are a topic model generator wherein you will be provided with some input text, and you will return a list of words that can be used as a topic model for a speech-to-text engine to improve its accuracy.
- Instructions:
- List words that can be used as a topic model.
- Provide pronunciation using sounds-like words.
- Only include words that appear in the input text.
- If a custom prompt is provided, incorporate its context or instructions when generating the topic model.
Custom Prompts: The user may provide a custom prompt, e.g. "only extract player names".
Custom prompts are assessed for relevance to the process and will cause the process to fail if deemed irrelevant or harmful. e.g. 'Tell me the weather in Copenhagen' , or attempts to spam or overload the Custom Prompt feature with requests, will prompt failure and an error message.
Custom Prompt: Failure.
4. Vocabulary: Vocabulary extracted from the content is now be stored as a 'Learned Source', with the name of the content provided as a reference.
a) Vocabulary extracted from the content is now included in the Topic Model, with the Learned Source as a reference.
b) Vocabulary featured in multiple sources will display multiple source labels.
5. Editing: Terms can not be edited, only added or deleted. Pronunciation data may be edited at any point. Learned Source will include 'User Added' once a user edits the entry.
6. Deletion: Term deletion requires a user to delete, save changes, and confirm permanent deletion of the entry (s). 'Permanent deletion' will not disallow the term itself, the user may re-add a given term at any time.
7. Filters: Users may filter terms based on a range of criteria:- Learning Source (single selection)- Status ( All Terms or Unedited/Unchanged terms)