APPLICATION DEVELOPER

The future is our choice
At Atos, as the global leader in secure and decarbonized digital, our purpose is to help design the future of the information space. Together we bring the diversity of our people’s skills and backgrounds to make the right choices with our clients, for our company and for our own futures.

• Develop and execute comprehensive test plans and strategies tailored to GenAI and RAG-based applications.
• Design and implement robust test cases and automation frameworks to ensure the effectiveness of AI models.

• Evaluate LLM responses for relevance, factual accuracy, and consistency.
• Identify and mitigate potential biases in the output, ensuring fairness and reliability.

• Test and validate high-quality retrieval processes from vector databases like FAISS, Pinecone, Weaviate, and ChromaDB to ensure the accurate context-based responses.

• Perform functional, integration, and regression testing to ensure AI-powered applications meet business requirements.
• Detect and resolve any issues related to functionality and model performance.

• Develop and implement automated API tests for validating AI models and vector search mechanisms.
• Ensure APIs are working as expected under various conditions and use cases.

• Evaluate the performance of LLMs using NLP metrics (e.g., BLEU, ROUGE, METEOR, perplexity) to assess the quality and effectiveness of model responses.

• Test AI/ML models to evaluate performance, latency, and response times under different workload conditions.
• Ensure systems can handle increasing scale while maintaining efficient performance.

• Ensure all AI applications comply with relevant data privacy, security standards, and regulations.
• Implement best practices to safeguard sensitive data and maintain data integrity throughout testing.

• Work closely with data scientists, machine learning engineers, and developers to share test results, identify areas for improvement, and refine AI models.
• Contribute to the continuous evolution of AI models based on test findings.

• Integrate AI testing processes into CI/CD pipelines for continuous testing and validation of AI models.
• Ensure seamless and efficient deployment of models through automated validation mechanisms.

• Implement techniques for generating synthetic data to enhance the robustness and performance of AI models.
• Ensure diverse, balanced, and realistic datasets for model training and evaluation.

• Use tools such as LangTest, PromptFoo, or LlamaIndex evals to test and validate prompt effectiveness and ensure they generate the desired outputs.

• Proven experience in AI/ML testing, particularly with Generative AI (GenAI) and Retrieval-Augmented Generation (RAG) models.
• Familiarity with NLP evaluation metrics such as BLEU, ROUGE, METEOR, and perplexity.
• Experience with CI/CD pipelines, model deployment, and continuous validation practices.
• Proficiency in using synthetic data generation techniques to improve model robustness.
• Experience working with tools like LangTest, PromptFoo, or LlamaIndex for prompt engineering and validation.
• Familiarity with data privacy and security standards in AI applications.
• Strong collaboration skills and ability to work with cross-functional teams (data scientists, ML engineers, developers).