LLMOps Engineer

ALX Africa is hiring an LLMOps engineer to own evaluation and reporting for its learner-facing AI products. The role focuses on building honest measurement systems around product behavior, including RAG quality, failure-mode analysis, and regression testing.

This is a good fit for someone who can decompose AI systems into measurable success criteria and help a product team understand what is actually working for learners.

• Strong Python skills and comfort analyzing messy model-generated data.
• Ability to decompose AI products into measurable success and failure metrics.
• Familiarity with tools such as Langfuse, RAGAS, DSPy, or similar evaluation stacks.
• Comfort building reporting loops and collaborating with product and engineering teams.
• Dashboarding, statistics, or cost-efficient eval design are useful additions.