OpenSTBench

v1.1.0 suspicious
5.0
Medium Risk

Unified multidimensional evaluation toolkit for S2TT and S2ST systems in offline and streaming speech translation settings

πŸ€– AI Analysis

Final verdict: SUSPICIOUS

The package shows some level of obfuscation and shell execution that require further scrutiny. Additionally, the sparse metadata about the author raises concerns about its reliability.

  • Code obfuscation present
  • Potential shell execution patterns
Per-check LLM notes
  • Network: No network calls detected.
  • Shell: Shell execution patterns observed may be related to the package's functionality but warrant further investigation to ensure there is no unintended behavior.
  • Obfuscation: The code shows signs of obfuscation with parts of the source code being obscured, but it does not definitively indicate malicious intent without further context.
  • Credentials: No patterns indicative of credential harvesting were detected.
  • Metadata: The author's information is sparse, indicating potential unreliability.

πŸ”¬ Heuristic Checks

βœ“ Outbound Network Calls

No suspicious network call patterns found

⚠ Code Obfuscation score 8.0

Found 4 obfuscation pattern(s)

  • model_source).to(self.device).eval() def _build_prompts(self, candidate_labels: Sequence[s
  • self.wavlm_model.eval() except Exception as exc: raise Runtime
  • f.utmos_model.to(self.device).eval() except Exception as e: print(f
  • .bleurt_model.to(self.device).eval() except Exception as e: print(f"❌ BLEUR
⚠ Shell / Subprocess Execution score 10.0

Found 5 shell execution pattern(s)

  • lurm job: {script_path}") os.system(f"sbatch {script_path}") sys.exit(0) class Visualizer:
  • turn try: subprocess.check_output("mfa version", shell=True) except Exception:
  • ) try: subprocess.run(cmd, shell=True, check=True) except Exception as e:
  • s.check_output("mfa version", shell=True) except Exception: print("Error: 'mfa'
  • subprocess.run(cmd, shell=True, check=True) except Exception as e: pri
βœ“ Credential Harvesting

No credential harvesting patterns detected

βœ“ Typosquatting

No typosquatting candidates detected

⚠ Registered Email Domain score 3.0

Suspicious email domain flags: Very short email domain: qq.com>

  • Very short email domain: qq.com>
βœ“ Suspicious Page Links

All external links appear legitimate

βœ“ Git Repository History

Repository sjtuayj/OpenSTBench appears legitimate

⚠ Maintainer History score 4.0

2 maintainer concern(s) found

  • Author name is missing or very short
  • Author "" appears to have only 1 package on PyPI (new or inactive account)
βœ“ Known CVE Vulnerabilities

No known vulnerabilities found in OSV database.

πŸ’‘ AI App Starter Prompt

Use this prompt to build a project with OpenSTBench
Your task is to develop a mini-application called 'SpeechTranslatePro' using the Python package 'OpenSTBench'. This tool will serve as a user-friendly interface for evaluating Speech-to-Text Translation (S2TT) and Speech-to-Speech Translation (S2ST) systems. The application should allow users to upload audio files, select between different translation models, and evaluate the output translations against ground truth data using metrics provided by OpenSTBench. Here’s a step-by-step guide on how to build it:

1. **Setup Environment**: Start by setting up your Python environment and installing OpenSTBench along with any other necessary packages like Flask for web framework.
2. **User Interface**: Create a simple yet intuitive web interface where users can upload their audio files, choose from available translation models (e.g., Google Translate API, Microsoft Azure Translation), and specify the source and target languages.
3. **Translation Process**: Integrate the selected translation model to process the uploaded audio file. Ensure that both offline and streaming modes of translation are supported based on user preference.
4. **Evaluation Module**: Implement an evaluation module using OpenSTBench to assess the quality of the translated output against the provided reference text. Include options to view detailed reports of evaluation metrics such as BLEU score, Word Error Rate (WER), etc.
5. **Results Display**: Design a section in the UI to display the results of the translation and evaluation. Users should be able to see the original text, translated text, and the corresponding evaluation scores.
6. **Customization Options**: Allow users to customize certain aspects of the evaluation process, such as choosing specific evaluation metrics or adjusting parameters for better accuracy.
7. **Documentation and Testing**: Write comprehensive documentation explaining how to use the application and how each feature works. Conduct thorough testing to ensure reliability and accuracy.

By following these steps, you will create a powerful and versatile tool for anyone interested in evaluating the performance of speech translation systems.