agentci-aadi

v0.2.1 suspicious
6.0
Medium Risk

Enterprise-Grade CI/CD Quality Gate for LLM Agents

🤖 AI Analysis

Final verdict: SUSPICIOUS

The package exhibits several suspicious characteristics including high shell and obfuscation risks, suggesting potential hidden functionalities or malicious intent. However, there is no definitive evidence of malicious activity.

  • High shell risk due to system command execution
  • Obfuscated code through dynamic imports and encoded strings
Per-check LLM notes
  • Network: Network calls to various API services suggest the package may be utilizing external services for functionality, which is not inherently malicious but requires scrutiny of its documentation and source code.
  • Shell: Executing system commands with ulimit settings can potentially modify system limits and behavior, indicating a higher risk of unintended side effects or potential misuse.
  • Obfuscation: The use of dynamic imports and encoded strings suggests an attempt to hide code logic, which is suspicious but not conclusively malicious without further context.
  • Credentials: No clear patterns indicating credential harvesting were found.
  • Metadata: The package shows signs of potential author inactivity and lack of maintainer history, raising concerns about its legitimacy and maintenance.

🔬 Heuristic Checks

⚠ Outbound Network Calls score 9.0

Found 6 network call pattern(s)

  • port httpx resp = httpx.post( "https://api.openai.com/v1/embeddings",
  • "openai": resp = httpx.get(url, headers={"Authorization": f"Bearer {key}"}, timeout=10)
  • nthropic": resp = httpx.get(url, headers={ "x-api-key": key,
  • "google": resp = httpx.get(f"{url}?key={key}", timeout=10) else: re
  • y.""" try: resp = httpx.get(f"{host}/api/tags", timeout=3) return resp.status_co
  • e: self._client = httpx.AsyncClient(timeout=60.0) return self._client async def clo
⚠ Code Obfuscation score 8.0

Found 4 obfuscation pattern(s)

  • ="Enable debug logging.") def eval(agent, scenarios, function, judges, runs, threshold, output,
  • chunk = __import__("json").loads(data) text = chunk.get("c
  • except __import__("json").JSONDecodeError: yield data
  • ={ **dict(__import__("os").environ), "PYTHONPATH": str(self.agent_
⚠ Shell / Subprocess Execution score 2.0

Found 1 shell execution pattern(s)

  • a ulimit result = subprocess.run( [sys.executable, str(harness_file), config]
✓ Credential Harvesting

No credential harvesting patterns detected

✓ Typosquatting

No typosquatting candidates detected

✓ Registered Email Domain

No author email provided

✓ Suspicious Page Links

All external links appear legitimate

⚠ Git Repository History score 3.0

Repository not found (deleted or private)

  • Repository not found (deleted or private)
⚠ Maintainer History score 4.0

2 maintainer concern(s) found

  • Author name is missing or very short
  • Author "" appears to have only 1 package on PyPI (new or inactive account)
✓ Known CVE Vulnerabilities

No known vulnerabilities found in OSV database.

💡 AI App Starter Prompt

Use this prompt to build a project with agentci-aadi 
Create a fully-functional mini-application that integrates the 'agentci-aadi' package to manage quality gates for language model agents within a Continuous Integration/Continuous Deployment (CI/CD) pipeline. Your application will serve as a bridge between your development environment and your CI/CD system, ensuring that every change adheres to strict quality standards before deployment.

### Project Scope:
1. **Setup**: Initialize a Python environment and install necessary packages including 'agentci-aadi'.
2. **Integration**: Integrate 'agentci-aadi' into your CI/CD pipeline to define quality gates for testing and validating LLM (Language Learning Model) agents.
3. **Features**:
   - Define customizable quality checks such as response accuracy, latency, and coherence.
   - Implement a dashboard to visualize the performance of each agent against these quality checks.
   - Provide logs and reports for failed tests to aid in debugging.
4. **User Interface**: Develop a simple web interface using Flask or Django to allow users to monitor and interact with the quality gate system.
5. **Testing**: Ensure that your application can handle multiple concurrent agents and different types of quality checks.
6. **Documentation**: Write comprehensive documentation on how to use the application, including setup instructions, configuration options, and usage examples.

### Utilization of 'agentci-aadi':
- Use 'agentci-aadi' to set up automated quality gates for your LLM agents.
- Leverage its capabilities to run predefined tests on agents during the CI/CD process.
- Employ it to generate detailed reports and logs that help in understanding the performance and issues faced by the agents.
- Integrate its feedback mechanisms into your application's workflow to automatically flag and address any issues detected during the quality check process.

This project aims to showcase the power of 'agentci-aadi' in enhancing the reliability and efficiency of LLM agents within enterprise environments.

💬 Discussion Feed

Leave a comment

No discussion yet. Be the first to share your thoughts!