Workbench v4.32.1

Schemathesis Workbench
Continuous fuzzing of open-source APIs.

ProfileBenchmark
1h budget per target, 30s per request, keeps testing after failures, no shrinking, logs show the targets' test credentials unmasked schemathesis run --max-time 3600 --request-timeout 30 --continue-on-failure --no-shrink --suppress-health-check all --output-sanitize false <SCHEMA_URL>

Schema coverage measured with TraceCov, code coverage with coverage.py

Totalsacross all targets
29
APIs Tested
20,442
Issues Found
7,006,110
Total Requests
53.1%
Avg Code Coverage
26h 8m 43s
Total Runtime
Targets29 APIs

Open WebUI

FastAPI OpenAPI 3.1 154k
332,690 requests
1h 0m 0s duration
Self-hosted AI chat interface for local and remote models
2,084

Issues Found

526 of 535 API operations affected

See Report
112
Server Errors
1
Security
1,971
Contract Violations

Coverage

Timeline
Lines42.3%
19,689 / 46,559
Branches18.9%
3,028 / 16,046
Operations98.5%
527 / 535
Parameters95.9%
680 / 709
Keywords97.2%
4,318 / 4,442

API Responses

56,873
2xx (17.1%)
261,901
4xx (78.7%)
13,906
5xx (4.2%)
Coverage Code | TraceCov
Stdout Web App | Schemathesis

Immich

NestJS OpenAPI 3.0.0 116k
335,492 requests
1h 0m 16s duration
Self-hosted photo and video management with mobile backup
893

Issues Found

277 of 279 API operations affected

See Report
42
Server Errors
849
Contract Violations
2
Other

Coverage

Timeline
Operations99.3%
277 / 279
Parameters98.0%
446 / 455
Keywords99.1%
4,189 / 4,227

API Responses

107,532
2xx (32.1%)
223,988
4xx (66.8%)
3,937
5xx (1.2%)
Coverage TraceCov
Stdout Web App | Schemathesis

Grafana

Macaron (in-tree fork) OpenAPI 3.0.3 77k
417,055 requests
1h 0m 0s duration
Observability platform for dashboards, alerting and data source queries
486

Issues Found

202 of 314 API operations affected

See Report
73
Server Errors
3
Security
409
Contract Violations
1
Other

Coverage

Timeline
Operations79.3%
249 / 314
Parameters81.8%
399 / 488
Keywords71.8%
1,355 / 1,886

API Responses

60,758
2xx (14.6%)
326,214
4xx (78.2%)
30,083
5xx (7.2%)
Coverage TraceCov
Stdout Web App | Schemathesis

LiteLLM

FastAPI OpenAPI 3.1.0 60k
221,636 requests
1h 0m 0s duration
Proxy server that serves 100+ LLM providers behind one OpenAI-compatible API, with keys, teams and budgets
3,247

Issues Found

871 of 905 API operations affected

See Report
529
Server Errors
5
Security
2,701
Contract Violations
12
Other

Coverage

Timeline
Lines43.5%
57,147 / 131,389
Branches22.5%
10,372 / 46,104
Operations99.7%
902 / 905
Parameters80.8%
990 / 1,225
Keywords70.0%
16,985 / 24,264

API Responses

26,301
2xx (11.9%)
172,471
4xx (77.8%)
22,864
5xx (10.3%)
Coverage Code | TraceCov
Stdout Web App | Schemathesis

Gitea

chi OpenAPI 2.0 58k
196,355 requests
1h 0m 0s duration
Self-hosted Git service with issues, pull requests, packages and CI
1,143

Issues Found

438 of 537 API operations affected

See Report
31
Server Errors
80
Security
910
Contract Violations
122
Other

Coverage

Timeline
Operations100.0%
537 / 537
Parameters100.0%
1,677 / 1,677
Keywords100.0%
3,106 / 3,106

API Responses

55,793
2xx (28.4%)
139,471
4xx (71.0%)
1,046
5xx (0.5%)
Coverage TraceCov
Stdout Web App | Schemathesis

Jellyfin

ASP.NET Core OpenAPI 3.0.4 58k
386,041 requests
1h 0m 1s duration
Self-hosted media server for movies, music and live TV
997

Issues Found

313 of 364 API operations affected

See Report
73
Server Errors
22
Security
897
Contract Violations
5
Other

Coverage

Timeline
Operations99.2%
361 / 364
Parameters94.8%
2,006 / 2,115
Keywords63.7%
17,934 / 28,167

API Responses

103,412
2xx (26.8%)
264,717
4xx (68.6%)
17,912
5xx (4.6%)
Coverage TraceCov
Stdout Web App | Schemathesis

Apache Airflow

FastAPI OpenAPI 3.1 47k
172,953 requests
1h 0m 0s duration
Platform to programmatically author, schedule and monitor workflows
332

Issues Found

90 of 128 API operations affected

See Report
45
Server Errors
283
Contract Violations
4
Other

Coverage

Timeline
Lines74.8%
8,605 / 11,498
Branches45.6%
1,298 / 2,844
Operations97.7%
125 / 128
Parameters94.9%
592 / 624
Keywords97.2%
2,379 / 2,447

API Responses

44,491
2xx (25.9%)
125,297
4xx (73.1%)
1,717
5xx (1.0%)
Coverage Code | TraceCov
Stdout Web App | Schemathesis

Paperless-ngx

Django REST Framework OpenAPI 3.0.3 46k
250,305 requests
1h 0m 2s duration
Document management system that scans, indexes and archives paper documents
776

Issues Found

161 of 163 API operations affected

See Report
72
Server Errors
1
Security
703
Contract Violations

Coverage

Timeline
Lines46.9%
8,788 / 18,742
Branches21.1%
1,075 / 5,106
Operations99.4%
162 / 163
Parameters84.1%
503 / 598
Keywords75.8%
7,155 / 9,439

API Responses

69,304
2xx (27.7%)
171,454
4xx (68.5%)
9,547
5xx (3.8%)
Coverage Code | TraceCov
Stdout Web App | Schemathesis

Qdrant

Actix Web OpenAPI 3.0.1 35k
208,722 requests
1h 0m 9s duration
Vector similarity search engine and vector database
127

Issues Found

57 of 69 API operations affected

See Report
10
Server Errors
116
Contract Violations
1
Other

Coverage

Timeline
Operations100.0%
69 / 69
Parameters80.6%
154 / 191
Keywords67.0%
5,966 / 8,898

API Responses

34,634
2xx (16.6%)
163,658
4xx (78.4%)
10,426
5xx (5.0%)
Coverage TraceCov
Stdout Web App | Schemathesis

PostgREST

Warp OpenAPI 2.0 28k
558,019 requests
1h 0m 0s duration
REST API generated from a PostgreSQL schema, serving a small library catalog
204

Issues Found

26 of 26 API operations affected

See Report
6
Server Errors
10
Security
188
Contract Violations

Coverage

Timeline
Operations96.3%
26 / 27
Parameters92.5%
149 / 161
Keywords89.4%
294 / 329

API Responses

195,563
2xx (35.0%)
362,354
4xx (64.9%)
102
5xx (0.0%)
Coverage TraceCov
Stdout Web App | Schemathesis

Firefly III

Laravel OpenAPI 3.0.0 25k
151,288 requests
1h 0m 0s duration
Personal finance manager with accounts, budgets, bills and rules
845

Issues Found

231 of 243 API operations affected

See Report
75
Server Errors
770
Contract Violations

Coverage

Timeline
Operations100.0%
243 / 243
Parameters84.3%
719 / 853
Keywords73.2%
3,018 / 4,122

API Responses

64,336
2xx (42.5%)
82,697
4xx (54.7%)
4,255
5xx (2.8%)
Coverage TraceCov
Stdout Web App | Schemathesis

Prefect

FastAPI OpenAPI 3.1 24k
58,085 requests
1h 0m 0s duration
Workflow orchestration framework for building resilient data pipelines in Python
507

Issues Found

172 of 187 API operations affected

See Report
49
Server Errors
455
Contract Violations
3
Other

Coverage

Timeline
Lines66.6%
14,522 / 21,820
Branches35.9%
1,961 / 5,468
Operations100.0%
187 / 187
Parameters96.1%
398 / 414
Keywords96.7%
16,301 / 16,852

API Responses

14,066
2xx (24.2%)
41,609
4xx (71.6%)
2,410
5xx (4.1%)
Coverage Code | TraceCov
Stdout Web App | Schemathesis

NetBox

Django REST Framework OpenAPI 3.0.3 22k
71,433 requests
1h 0m 0s duration
Network source of truth for modelling and documenting infrastructure
2,077

Issues Found

1,217 of 1,255 API operations affected

See Report
48
Server Errors
131
Security
1,898
Contract Violations

Coverage

Timeline
Lines66.7%
39,885 / 59,785
Branches14.8%
1,638 / 11,042
Operations99.9%
1,255 / 1,256
Parameters53.9%
10,290 / 19,087
Keywords22.6%
34,953 / 154,759

API Responses

11,079
2xx (15.5%)
59,258
4xx (83.0%)
1,096
5xx (1.5%)
Coverage Code | TraceCov
Stdout Web App | Schemathesis

OpenProject

Rails OpenAPI 3.1.2 16k
398,443 requests
1h 0m 0s duration
Project management with work packages, boards, time tracking and meetings
1,223

Issues Found

317 of 320 API operations affected

See Report
112
Server Errors
1
Security
1,110
Contract Violations

Coverage

Timeline
Operations100.0%
320 / 320
Parameters94.6%
453 / 479
Keywords91.1%
2,392 / 2,627

API Responses

42,793
2xx (10.7%)
340,474
4xx (85.5%)
15,176
5xx (3.8%)
Coverage TraceCov
Stdout Web App | Schemathesis

Unleash

Express OpenAPI 3.0.3 14k
135,817 requests
1h 0m 9s duration
Feature flag management with projects, environments, strategies and segments
517

Issues Found

154 of 211 API operations affected

See Report
59
Server Errors
3
Security
444
Contract Violations
11
Other

Coverage

Timeline
Operations100.0%
211 / 211
Parameters99.7%
334 / 335
Keywords99.9%
2,257 / 2,260

API Responses

55,069
2xx (40.6%)
76,198
4xx (56.1%)
4,467
5xx (3.3%)
Coverage TraceCov
Stdout Web App | Schemathesis

Mealie

FastAPI OpenAPI 3.0 13k
129,224 requests
1h 0m 0s duration
Self-hosted recipe manager and meal planner with a RestAPI backend
772

Issues Found

232 of 247 API operations affected

See Report
104
Server Errors
668
Contract Violations

Coverage

Timeline
Lines74.5%
13,046 / 17,510
Branches44.6%
1,551 / 3,476
Operations96.4%
238 / 247
Parameters91.3%
679 / 744
Keywords91.5%
5,080 / 5,553

API Responses

49,282
2xx (38.1%)
69,506
4xx (53.8%)
10,435
5xx (8.1%)
Coverage Code | TraceCov
Stdout Web App | Schemathesis

OpenReplay

FastAPI OpenAPI 3.1.0 13k
342,286 requests
1h 0m 0s duration
Session replay and product analytics platform
508

Issues Found

142 of 147 API operations affected

See Report
78
Server Errors
429
Contract Violations
1
Other

Coverage

Timeline
Lines0.0%
0 / 8,109
Operations98.0%
144 / 147
Parameters99.4%
180 / 181
Keywords99.1%
767 / 774

API Responses

108,269
2xx (31.6%)
216,163
4xx (63.2%)
17,842
5xx (5.2%)
Coverage Code | TraceCov
Stdout Web App | Schemathesis

Polar

FastAPI OpenAPI 3.1 10k
114,434 requests
1h 0m 1s duration
Turn your software into a business - payment and monetization platform for open source
621

Issues Found

212 of 229 API operations affected

See Report
95
Server Errors
6
Security
519
Contract Violations
1
Other

Coverage

Timeline
Lines66.1%
19,979 / 30,218
Branches31.1%
1,954 / 6,282
Operations96.9%
222 / 229
Parameters59.4%
303 / 510
Keywords84.2%
4,600 / 5,465

API Responses

25,842
2xx (22.8%)
85,268
4xx (75.3%)
2,147
5xx (1.9%)
Coverage Code | TraceCov
Stdout Web App | Schemathesis

InvenTree

Django REST Framework OpenAPI 3.0 7.7k
35,106 requests
1h 0m 0s duration
Open source inventory management system with a Django REST Framework API
1,383

Issues Found

551 of 583 API operations affected

See Report
104
Server Errors
7
Security
1,272
Contract Violations

Coverage

Timeline
Lines60.0%
22,619 / 37,680
Branches24.7%
2,392 / 9,692
Operations98.6%
575 / 583
Parameters44.9%
986 / 2,197
Keywords40.6%
6,743 / 16,598

API Responses

6,029
2xx (17.2%)
27,442
4xx (78.2%)
1,635
5xx (4.7%)
Coverage Code | TraceCov
Stdout Web App | Schemathesis

Weblate

Django REST Framework OpenAPI 3.1 6.1k
185,586 requests
1h 0m 0s duration
Web-based continuous localization system with a Django REST Framework API
355

Issues Found

125 of 160 API operations affected

See Report
18
Server Errors
317
Contract Violations
20
Other

Coverage

Timeline
Lines48.2%
23,389 / 48,512
Branches19.5%
2,764 / 14,168
Operations90.1%
145 / 161
Parameters83.6%
499 / 597
Keywords77.2%
4,560 / 5,905

API Responses

47,711
2xx (25.7%)
136,657
4xx (73.6%)
1,218
5xx (0.7%)
Coverage Code | TraceCov
Stdout Web App | Schemathesis

OPAL

FastAPI OpenAPI 3.1.0 5.5k
266,564 requests
1h 0m 0s duration
Administration layer that keeps policy agents such as OPA and Cedar up to date in real time
80

Issues Found

17 of 20 API operations affected

See Report
13
Server Errors
64
Contract Violations
3
Other

Coverage

Timeline
Lines57.7%
3,372 / 5,845
Branches29.1%
355 / 1,218
Operations100.0%
20 / 20
Parameters97.3%
36 / 37
Keywords99.6%
256 / 257

API Responses

88,616
2xx (33.2%)
102,926
4xx (38.6%)
75,022
5xx (28.1%)
Coverage Code | TraceCov
Stdout Web App | Schemathesis

Dependency-Track

Jersey OpenAPI 3.0.1 4.3k
566,559 requests
1h 0m 0s duration
Software composition analysis platform that tracks components and vulnerabilities from SBOMs
520

Issues Found

196 of 219 API operations affected

See Report
95
Server Errors
3
Security
417
Contract Violations
5
Other

Coverage

Timeline
Operations100.0%
219 / 219
Parameters92.3%
550 / 596
Keywords10.6%
1,182 / 11,101

API Responses

133,116
2xx (23.6%)
406,964
4xx (72.2%)
23,370
5xx (4.1%)
Coverage TraceCov
Stdout Web App | Schemathesis

Kuma

go-restful OpenAPI 3.1.0 4.0k
501,324 requests
1h 0m 0s duration
Service mesh control plane for Kubernetes and VMs
383

Issues Found

139 of 141 API operations affected

See Report
383
Contract Violations

Coverage

Timeline
Operations100.0%
141 / 141
Parameters99.2%
364 / 367
Keywords99.0%
3,548 / 3,585

API Responses

293,620
2xx (58.6%)
207,539
4xx (41.4%)
0
5xx (0.0%)
Coverage TraceCov
Stdout Web App | Schemathesis

pygeoapi

Flask OpenAPI 3.0 626
200,655 requests
1h 0m 0s duration
OGC API server for geospatial data, built on Flask
188

Issues Found

61 of 66 API operations affected

See Report
14
Server Errors
174
Contract Violations

Coverage

Timeline
Lines61.1%
3,417 / 5,590
Branches46.2%
769 / 1,664
Operations98.5%
66 / 67
Parameters88.8%
190 / 214
Keywords93.3%
376 / 403

API Responses

81,377
2xx (40.6%)
116,610
4xx (58.1%)
2,668
5xx (1.3%)
Coverage Code | TraceCov
Stdout Web App | Schemathesis

Openverse

Django REST Framework OpenAPI 3.0 380
122,441 requests
1h 0m 0s duration
Search engine API for openly licensed images and audio
30

Issues Found

13 of 17 API operations affected

See Report
2
Server Errors
28
Contract Violations

Coverage

Timeline
Lines78.5%
2,983 / 3,801
Branches51.8%
325 / 628
Operations100.0%
17 / 17
Parameters100.0%
70 / 70
Keywords100.0%
240 / 240

API Responses

57,770
2xx (47.2%)
64,586
4xx (52.8%)
74
5xx (0.1%)
Coverage Code | TraceCov
Stdout Web App | Schemathesis

Covid 19 Japan Web API

Flask OpenAPI 2.0 129
3,489 requests
2m 0s duration
Web API to get COVID-19 information of each prefecture in Japan
1

Issue Found

1 of 4 API operations affected

See Report
1
Contract Violations

Coverage

Timeline
Lines100.0%
84 / 84
Branches100.0%
8 / 8
Operations100.0%
4 / 4
Parameters100.0%
3 / 3
Keywords100.0%
3 / 3

API Responses

3,331
2xx (95.5%)
158
4xx (4.5%)
0
5xx (0.0%)
Coverage Code | TraceCov
Stdout Web App | Schemathesis

aeman

net/http OpenAPI 3.0.3 34
630,498 requests
1h 0m 0s duration
Sprint and planning board for engineering teams, stored as commits in a git repository
121

Issues Found

51 of 60 API operations affected

See Report
14
Server Errors
107
Contract Violations

Coverage

Timeline
Operations98.4%
60 / 61
Parameters100.0%
164 / 164
Keywords100.0%
361 / 361

API Responses

195,004
2xx (30.9%)
411,857
4xx (65.3%)
23,637
5xx (3.7%)
Coverage TraceCov
Stdout Web App | Schemathesis

REST-ler Blog API

Flask OpenAPI 2.0
5,481 requests
5m 0s duration
Demo API used in the original REST-ler paper
22

Issues Found

5 of 6 API operations affected

See Report
3
Server Errors
19
Contract Violations

Coverage

Timeline
Lines94.0%
173 / 184
Branches66.7%
12 / 18
Operations100.0%
6 / 6
Parameters100.0%
5 / 5
Keywords100.0%
6 / 6

API Responses

3,662
2xx (66.8%)
1,693
4xx (30.9%)
126
5xx (2.3%)
Coverage Code | TraceCov
Stdout Web App | Schemathesis

Sample API

FastAPI OpenAPI 3.1.0
8,129 requests
1m 0s duration
A sample API designed to test the coverage effectiveness of Schemathesis tests
0

Issues Found

0 of 1 API operations affected

See Report

Coverage

Timeline
Lines100.0%
5 / 5
Operations100.0%
1 / 1

API Responses

8,124
2xx (99.9%)
5
4xx (0.1%)
0
5xx (0.0%)
Coverage Code | TraceCov
Stdout Web App | Schemathesis

Ready to test your API?

Start finding bugs in minutes with a single command

uvx schemathesis run <YOUR API SPEC URL>
Get Started Documentation

Want the same schema coverage for your own tests? Try TraceCov