RLHF Specialist

About the role

The RLHF Specialist will help improve large language models by designing and operating Reinforcement Learning from Human Feedback pipelines. Working remotely with a global team, you will ensure that AI outputs are helpful, honest, and harmless.

Key responsibilities

• Generate high‑quality preference data by ranking model responses on helpfulness, honesty, and harmlessness.
• Design multi‑turn prompts to stress‑test model reasoning and safety.
• Write chain‑of‑thought explanations to train reward models.
• Collaborate with ML engineers to analyse failure modes and close data gaps.
• Develop and iterate annotation strategies for consistent preference scoring.
• Probe models for biases, hallucinations, and vulnerabilities, documenting findings.
• Analyze edge cases where reward models behave unexpectedly and suggest data interventions.
• Create templated instruction sets for large annotation teams and translate RL concepts into repeatable tasks.
• Maintain a personal test set of prompts to monitor model performance over time.

Required profile

• At least 2 years of experience in data annotation, model evaluation, computational linguistics, or AI trust & safety.
• Strong proficiency in Python and deep‑learning frameworks (PyTorch, JAX, or TensorFlow).
• Deep understanding of reinforcement‑learning concepts such as PPO, trust regions, and reward hacking.
• Hands‑on experience fine‑tuning open‑source models (e.g., Llama 2/3, Mistral, Gemma) using LoRA/QLoRA.
• Experience with annotation platforms (LabelBox, Scale AI, Snorkel) and human‑in‑the‑loop workflows.
• Familiarity with cloud AI services (AWS SageMaker, GCP Vertex AI).

Required skills

• Python
• PyTorch
• JAX
• TensorFlow
• PPO, Trust Regions, Reward Hacking
• LoRA / QLoRA
• Llama 2, Llama 3, Mistral, Gemma
• LabelBox, Scale AI, Snorkel
• AWS SageMaker
• GCP Vertex AI