Gen AI Analyst

ActiveFence is seeking a driven, detail-focused professional to become a vital part of our team as a Generative AI Analyst. In this role, you'll dive into the cutting-edge of technology, meticulously analyzing various content infringements to secure the new wave of Generative AI tools. Your duties will include collaborating with experts in diverse fields such as Hate Speech, Misinformation, Intellectual Property and Copyright, Child Safety, among others.

Your tasks will involve writing adversarial; prompts to identify weaknesses in various AI models, including Large Language Models (LLMs), Text-to-Image, Text-to-Video, and beyond. You'll also oversee data management to guarantee the highest quality of outputs.

Responsibilities:

• Developing adversarial and risky prompts across several areas of abuse to expose potential vulnerabilities in models.
• Handling extensive datasets across multiple languages and areas of abuse, ensuring precision and meticulous attention to detail.
• Ongoing investigation into new tactics for circumventing foundational models' safety measures.
• Working alongside diverse teams—engineering, product, policy—to tackle new challenges and craft forward-thinking strategies and resolutions.
• Promoting a culture of knowledge exchange and continual learning within the team.

Requirements:

Must have:

• Familiarity with Generative AI models is essential, though direct technical experience is not a prerequisite.
• Command of English at a near-native level.
• Attention to detail, organizational capabilities, and the capacity to juggle numerous tasks concurrently.
• Data analysis

Additional Wants:

• Experience with various model types (Text-to-Text, Text-to-Image) is desirable.
• Prior experience with OSINT (Open Source Intelligence) will be considered an asset.
• A self-starter attitude, with the energy to excel in a fast-moving and variable environment.

ActiveFence is the leading tool stack for Trust & Safety teams, worldwide. By relying on ActiveFence’s end-to-end solution, Trust & Safety teams – of all sizes – can keep users safe from the widest spectrum of online harms, unwanted content, and malicious behavior, including child safety, disinformation, fraud, hate speech, terror, nudity, and more. 

Using cutting-edge AI and a team of world-class subject-matter experts to continuously collect, analyze, and contextualize data, ActiveFence ensures that in an ever-changing world, customers are always two steps ahead of bad actors. As a result, Trust & Safety teams can be proactive and provide maximum protection to users across a multitude of abuse areas, in 70+ languages. 

Backed by leading Silicon Valley investors such as CRV and Norwest, ActiveFence has raised $100M to date; employs 300 people worldwide, and has contributed to the online safety of billions of users across the globe.