Installation Guide

This guide provides detailed installation instructions for KARMA on different platforms and environments.

Installation Methods

KARMA can be installed through pip and also uv

Installing UV

Run this command

curl -LsSf https://astral.sh/uv/install.sh | sh

Or see UV docs here - https://docs.astral.sh/uv/getting-started/installation/

pip

pip install karma-medeval

uv

uv pip install karma-medeval

Or add to your uv project through

uv add karma-medeval

Verify installation

karma --version

Optional Dependencies

Audio Support

For audio-related datasets and ASR metrics:

pip

pip install "karma-medeval[audio]"

uv

uv pip install "karma-medeval[audio]"

This includes:

• jiwer - Word Error Rate calculations
• num2words - Number to word conversion
• torchaudio - Audio processing

Install with all dependencies

pip

Install the development dependencies through pip

pip install "karma-medeval[all]"

uv

uv pip install "karma-medeval[all]"

Development installation

Clone the repository

# Clone the repository
git clone https://github.com/eka-care/KARMA-OpenMedEvalKit.git
cd KARMA-OpenMedEvalKit

Install developer version

Then run either of these commands for developer installation

pip

Install the development dependencies through pip

pip install -e .

uv

uv sync

Install with all dependencies

pip

Install the development dependencies through pip

pip install -e ".[all]"

uv

uv sync --all-extras

Environment Configuration

Create a .env file in your project root:

# Required: HuggingFace token for model downloads
HUGGINGFACE_TOKEN=your_token_here

# Cache configuration
KARMA_CACHE_TYPE=duckdb
KARMA_CACHE_PATH=./cache.db

# Logging
LOG_LEVEL=INFO

# Optional: OpenAI API key (for certain metrics)
OPENAI_API_KEY=your_openai_key

# Optional: DynamoDB configuration (for production)
# KARMA_CACHE_TYPE=dynamodb
# AWS_REGION=us-east-1
# DYNAMODB_TABLE_NAME=karma-cache

HuggingFace Token

To access gated models or datasets, set this environment variable with your Huggingface token.

You can see the guide to create tokens here

# Login to HuggingFace
huggingface-cli login

# Or set environment variable
export HUGGINGFACE_TOKEN=your_token_here