assayer

v1.0.1 suspicious
5.0
Medium Risk

Run a prompt across multiple LLMs and compare outputs side by side in the terminal.

🤖 AI Analysis

Final verdict: SUSPICIOUS

The package shows some red flags, particularly concerning metadata and maintainership, which could suggest potential risks. However, the network and shell risks are relatively low.

  • Missing maintainer information
  • No associated Git repository
Per-check LLM notes
  • Network: The network calls appear to be local and may be part of the package's functionality, but further investigation into the API endpoint is needed.
  • Shell: No shell execution patterns detected.
  • Metadata: The package has some red flags such as missing maintainer information and no associated Git repository, which could indicate potential issues.

📦 Package Quality Overall: Low (4.4/10)

✦ High Test Suite 9.0

Test suite present — 6 test file(s) found

  • Test runner config found: pyproject.toml
  • 6 test file(s) detected (e.g. test_cli.py)
◈ Medium Documentation 5.0

Some documentation present

  • Detailed PyPI description (4965 chars)
○ Low Contributing Guide 2.0

No contributing guide or governance files found

  • No CONTRIBUTING, CODE_OF_CONDUCT, or governance files found
◈ Medium Type Annotations 5.0

Partial type annotation coverage

  • 35 type-annotated function signatures detected in source
○ Low Multiple Contributors 1.0

Unable to verify contributor count: no GitHub repository found

  • No GitHub repository linked — contributor count unavailable

🔬 Heuristic Checks

⚠ Outbound Network Calls score 4.5

Found 3 network call pattern(s)

  • try: response = httpx.get("http://localhost:11434/api/tags", timeout=3.0) res
  • try: async with httpx.AsyncClient(timeout=3.0) as client: response = await client
  • try: async with httpx.AsyncClient(timeout=60.0) as client: response = await c
✓ Code Obfuscation

No obfuscation patterns detected

✓ Shell / Subprocess Execution

No shell execution patterns detected

✓ Credential Harvesting

No credential harvesting patterns detected

✓ Typosquatting

No typosquatting candidates detected

✓ Registered Email Domain

No author email provided

✓ Suspicious Page Links

All external links appear legitimate

✓ Git Repository History

No GitHub repository linked

  • No GitHub repository link found
⚠ Maintainer History score 4.0

2 maintainer concern(s) found

  • Author name is missing or very short
  • Author "" appears to have only 1 package on PyPI (new or inactive account)
✓ Known CVE Vulnerabilities

No known vulnerabilities found in OSV database.

💡 AI App Starter Prompt

Use this prompt to build a project with assayer 
Create a mini-application called 'LLMComparer' that leverages the 'assayer' Python package to facilitate comparison of responses from different large language models (LLMs). This application should allow users to input a query or prompt and then display the output from multiple LLMs side by side in the terminal for easy comparison. Here are the key steps and features to implement:

1. **Setup**: Ensure all necessary packages, including 'assayer', are installed and properly configured.
2. **User Interface**: Design a simple command-line interface where users can enter their prompts.
3. **Prompt Processing**: Allow users to input custom prompts or select from predefined ones.
4. **LLM Selection**: Provide options for users to choose which LLMs they want to use for their comparisons (e.g., GPT-3, Claude).
5. **Response Display**: Implement functionality to display the responses from selected LLMs side by side in the terminal for direct comparison.
6. **Output Formatting**: Enhance readability by formatting the output in a clear and organized manner.
7. **Additional Features**: Consider adding features like saving the comparison results to a file, allowing users to rate or comment on the responses, and providing statistics about response times.
8. **Testing and Documentation**: Thoroughly test the application and provide clear documentation for setup and usage.

💬 Discussion Feed

Leave a comment

No discussion yet. Be the first to share your thoughts!