Sarah Mcivor

Key Skills

Software

Data Annotation Tech

Top Subject Matter

No subject matter listed

Top Data Types

Audio

Document

Image

Top Task Types

Bounding Box

Evaluation Rating

Text Generation

Freelancer Overview

I've worked extensively across projects for one of the top tech companies in the world, working with their models to refine their personal assistant and its consumer interaction across voice and audio, online suite and mobile assistance for text generation, task setting, data retrieval from documents or emails, to assisting with email creation. I'm used to working with LLMs to analyse and improve their responses to prompts, making amendments as required to aid in natural language processing. Additionally, I've worked specifically on adversarial projects both to give prompts to LLMs to try to trick them into providing adversarial content, as well as identifying and flagging it when provided outside of the prompt context. I have also worked in image identification and classification, audio clarity and creative projects for storytelling and poetry, as well as data retrieval evaluation.

Labeling Experience

Action recognition

Data Annotation TechTextAction Recognition

This was an extensive project for a mobile personal assistant to assist the user in basic functions where they have put in a text or verbal prompt and the assistant responded in text by attempting to complete the task and giving an appropriate response. Some tasks were straightforward and easily completed, but sometimes the models would not perform tasks such as correctly setting timers, or would give capability refusals when the prompt was for something like "remind me at 3 o'clock to buy milk," therefore I would make notes that this is something that the model should do by setting a reminder for that day and confirming to the user that they have done so.

This was an extensive project for a mobile personal assistant to assist the user in basic functions where they have put in a text or verbal prompt and the assistant responded in text by attempting to complete the task and giving an appropriate response. Some tasks were straightforward and easily completed, but sometimes the models would not perform tasks such as correctly setting timers, or would give capability refusals when the prompt was for something like "remind me at 3 o'clock to buy milk," therefore I would make notes that this is something that the model should do by setting a reminder for that day and confirming to the user that they have done so.

2024 - 2024

Video assistant

Data Annotation TechVideoQuestion Answering

I worked with models to assess their awareness of objects in a real space during a conversation with a user. For example, if the user was using the camera and asked the model how to put together a chair which is in various pieces, I would ensure that the model would check whether they have the instructions and all parts available and gives helpful, accurate, step-by-step responses to guide the user. Or if the user showed a video of a black pug playing with a ball, I would make sure that the model has correctly responded that the animal it sees is, at the very least, a "small black dog" rather than a "dog" or wrongly a "cat." I would rate the quality of responses to the questions for accuracy, length, safety and also how realistic they sounded, as the aim was to feel like natural language rather than robotic AI.

I worked with models to assess their awareness of objects in a real space during a conversation with a user. For example, if the user was using the camera and asked the model how to put together a chair which is in various pieces, I would ensure that the model would check whether they have the instructions and all parts available and gives helpful, accurate, step-by-step responses to guide the user. Or if the user showed a video of a black pug playing with a ball, I would make sure that the model has correctly responded that the animal it sees is, at the very least, a "small black dog" rather than a "dog" or wrongly a "cat." I would rate the quality of responses to the questions for accuracy, length, safety and also how realistic they sounded, as the aim was to feel like natural language rather than robotic AI.

2024 - 2024

Audio quality checking

Data Annotation TechAudioEvaluation Rating

A project working with models who produced audio responses to a spoken word user input. My role was to compare the responses of two models and choose the better of the two, rating them for accuracy, safety, length and other quality rankings, occasionally suggesting alternative scripts.

A project working with models who produced audio responses to a spoken word user input. My role was to compare the responses of two models and choose the better of the two, rating them for accuracy, safety, length and other quality rankings, occasionally suggesting alternative scripts.

2024 - 2024

Translation

Data Annotation TechTextTranslation Localization

Checking of prompts and responses in other languages and dialects using Google Translate to ensure the safety of responses.

Checking of prompts and responses in other languages and dialects using Google Translate to ensure the safety of responses.

2024 - 2024

Chatbot conversations / prompt and response amending

Data Annotation TechTextPrompt Response Writing SFT

I worked with models to hold conversations where I would write prompts, assess the responses and where needed either edit or write more appropriate responses to the prompt. The subject matter varied from giving unsafe or misleading prompts to attempt to illicit an unsafe response to correct, to creative writing or fact checking.

I worked with models to hold conversations where I would write prompts, assess the responses and where needed either edit or write more appropriate responses to the prompt. The subject matter varied from giving unsafe or misleading prompts to attempt to illicit an unsafe response to correct, to creative writing or fact checking.

2024 - 2024

Show more

Education

Work History