OraFinite

OraFinite is a self-hosted AI security platform for LLM applications. The repository contains a Rust/Axum API server, Python ML sidecar wrapping LLM Guard and Garak, gRPC protobuf boundary, PostgreSQL migrations, Redis cache/pub-sub/rate limiting, Nginx reverse proxy, Docker Compose GPU-only deployment, and a Next.js dashboard with Better Auth, model registry, guard/scanner pages, API keys, logs, SSE monitoring, subscriptions, and eSewa payment integration.

LLM applications need protection against prompt injection, toxic outputs, data leakage, PII/secrets exposure, and model endpoint vulnerabilities, but stitching scanning libraries into product infrastructure is operationally complex.

Separate application/API concerns from ML scanning workloads: Rust handles auth, API keys, rate limits, persistence, events, and orchestration; Python handles GPU-bound LLM Guard/Garak scanning; gRPC connects the services; Redis/PostgreSQL/Nginx/Docker Compose provide infrastructure boundaries.

• Rust/Axum API uses SQLx/PostgreSQL, Redis, JWT, API-key middleware, rate limiting, and event/log endpoints.
• Python ML sidecar wraps LLM Guard real-time scanning and Garak vulnerability scanning behind gRPC.
• Docker Compose defines Nginx, frontend, Rust server, PostgreSQL, Redis, and GPU-only ML sidecar services.
• Next.js dashboard includes model registry, guard configuration, scanner flows, credentials, logs, SSE guard-event monitoring, and Better Auth.
• Architecture demonstrates service boundaries between API gateway, persistence, cache/events, and CUDA inference workloads.

OraFinite is presented as a hackathon project. The portfolio describes repository-verified architecture and product scope without implying sole ownership of team work.