Model Safety Checker

Inspect a model repository for code that runs on load and weights that have been tampered with.

What this is

Downloading a model from a hub means running someone else's file on your computer. Some formats execute code the moment you load them; some configs pull in Python from the repository; and weights themselves can carry behaviour nobody advertised. Paste a repository below and it is inspected in place.

Two kinds of check

Deterministic Static analysers run over the files. For example, if a file imports os.system, it will run that command when loaded. Expected false-positive rate: zero.
Statistical Judgements with a measured false-positive rate. They compare a model against clean ones and flag what stands out.
A clean result means these checks found nothing. That is not the same as the model being safe. Every check states what it examined and, when it couldn't run, why.
Deep compares the model against the base it declares, so it only applies to fine-tunes and adapters — a base model like gpt2 has nothing to be compared with. It fetches embedding matrices and attention weights: minutes the first time, then cached for every model sharing that base.
Try:
Starting

What this scanner looks for

Malicious pickle files in PyTorch models

pytorch_model.bin, .pt and .ckpt files are Python pickles. Loading one with torch.load executes the opcodes inside it, so a crafted checkpoint can run arbitrary commands — the classic payload imports os.system or subprocess.Popen. This scanner walks the opcode stream without executing it and reports every import the file would perform.

Models that run code through trust_remote_code

A repository needs no pickle at all to execute code. If config.json contains an auto_map, loading the model with trust_remote_code=True imports Python modules from the repository itself. The scanner reports those entries and analyses the modules they point at.

Malformed safetensors files

The safetensors format executes nothing, which removes the pickle problem but leaves the parser. Tensor byte ranges that overlap, run past the end of the file, or declare an implausible header size are reported.

Backdoor signatures in LoRA adapters and fine-tuned weights

Beyond executable payloads, weights themselves can carry behaviour that is not advertised. The statistical checks compare an adapter or fine-tune against the base model it declares, measuring the spectral concentration of the weight update and how far individual token embeddings moved.

Common questions

Does it download the model?

No. It reads the repository file listing, file headers and selected byte ranges. Deep scans additionally fetch the embedding matrix and attention projections needed for comparison against the base model.

Is a clean result a guarantee?

No. A clean result means these checks found nothing. Every check states what it examined, and when it could not run, why.

What does it cost?

Nothing. It reads public repositories on the Hugging Face hub.

Can I use it from a script?

Yes. The same scans are available over HTTP, free and without a key: POST /api/scan with {"repo": "owner/model", "deep": true} returns a job id, and GET /api/scan/{id} returns the report once it finishes. GET /api/checks lists every check and what it does. Interactive documentation is at /docs, with the OpenAPI schema at /openapi.json.

Scans are queued and run two at a time, so a burst of requests will wait rather than fail. Please keep automated use reasonable — this runs on a single small machine.