LLM Evaluation Hub

• Automated Vulnerability Detection:
  ‍Giskard not only tests your AI, but also automatically detects critical vulnerabilities such as hallucinations and security flaws. Since test cases can be virtually endless and highly domain-specific, Giskard leverages both internal and external data sources (e.g., RAG knowledge bases) to automatically and exhaustively generate test cases.‍
• Proactive Monitoring:
  At Giskard, we believe itʼs too late if issues are only discovered by users once the system is in production. Thatʼs why we focus on proactive monitoring, providing tools to detect AI vulnerabilities before they surface in real-world use. This involves continuously generating different attack scenarios and potential hallucinations throughout your AIʼs lifecycle.‍
• Accessible for Business Stakeholders:
  Giskard is not just a developer tool—itʼs also designed for business users like domain experts and product managers. It offers features such as a collaborative red-teaming playground and annotation tools, enabling anyone to easily craft test cases.

Giskard employs various methods to detect vulnerabilities, depending on their type:

• Internal Knowledge:
  Leveraging company expertise (e.g., RAG knowledge base) to identify hallucinations.
• Security Vulnerability Taxonomies:
  Detecting issues such as stereotypes, discrimination, harmful content, personal information disclosure, prompt injections, and more.
• External Resources:
  Using cybersecurity monitoring and online data to continuously identify new vulnerabilities.
• Internal Prompt Templates:
  Applying templates based on our extensive experience with various clients.

Giskard can be used before and after deployment:

• Before deployment:
  Provides comprehensive quantitative KPIs to ensure your AI agent is production-ready.
• After deployment:
  Continuously detects new vulnerabilities that may emerge once your AI application is in production.

Yes! After subscribing to the Giskard Hub, you can opt for support from our LLM researchers to help mitigate vulnerabilities. We can also assist in designing effective safeguards in production.

The Giskard Hub supports all types of text-to-text conversational bots.

Giskard operates as a black-box testing tool, meaning the Hub does not need to know the internal components of your agent (foundational models, vector database, etc.).

The bot as a whole only needs to be accessible through an API endpoint.

• Giskard Open Source → A Python library intended for developers.
• LLM Hub → An enterprise solution offering a broader range of features such as:
  • A red-teaming playground
  • Cybersecurity monitoring and alerting
  • An annotation studio
  • More advanced security vulnerability detection

For a complete overview of LLM Hub’s features, follow this link.

Yes, you can easily install the Giskard Hub on your internal machines or private cloud.

The Giskard Hub is available through annual subscription based on the number of AI systems.

For pricing details, please follow this link.