Workbench v4.30.1

Schemathesis Workbench
Continuous fuzzing of open-source APIs.

ProfileBenchmark
1h budget per target, keeps testing after failures, no shrinking schemathesis run --max-time 3600 --continue-on-failure --no-shrink --suppress-health-check all <SCHEMA_URL>

Schema coverage measured with TraceCov, code coverage with coverage.py

Totalsacross all targets
29
APIs Tested
20,116
Issues Found
6,075,311
Total Requests
53.6%
Avg Code Coverage
23h 59m 38s
Total Runtime
Targets29 APIs

Open WebUI

FastAPI OpenAPI 3.1 154k
335,638 requests
1h 0m 0s duration
Self-hosted AI chat interface for local and remote models
2,097

Issues Found

527 of 535 API operations affected

See Report
120
Server Errors
2
Security
1,975
Contract Violations

Coverage

Timeline
Lines40.4%
18,827 / 46,559
Branches17.1%
2,739 / 16,046
Operations98.7%
528 / 535
Parameters95.1%
674 / 709
Keywords94.0%
4,177 / 4,442

API Responses

66,192
2xx (19.7%)
254,039
4xx (75.7%)
15,293
5xx (4.6%)
Coverage Code | TraceCov
Stdout Web App | Schemathesis

Immich

NestJS OpenAPI 3.0.0 116k
287,098 requests
1h 0m 0s duration
Self-hosted photo and video management with mobile backup
908

Issues Found

277 of 279 API operations affected

See Report
51
Server Errors
1
Security
854
Contract Violations
2
Other

Coverage

Timeline
Operations99.3%
277 / 279
Parameters98.0%
446 / 455
Keywords97.6%
4,124 / 4,227

API Responses

123,460
2xx (43.0%)
158,362
4xx (55.2%)
5,123
5xx (1.8%)
Coverage TraceCov
Stdout Web App | Schemathesis

Grafana

Macaron (in-tree fork) OpenAPI 3.0.3 77k
38,299 requests
5m 0s duration
Observability platform for dashboards, alerting and data source queries
509

Issues Found

239 of 314 API operations affected

See Report
56
Server Errors
3
Security
450
Contract Violations

Coverage

Timeline
Operations100.0%
314 / 314
Parameters99.4%
485 / 488
Keywords97.4%
1,837 / 1,886

API Responses

5,058
2xx (13.2%)
31,163
4xx (81.4%)
2,077
5xx (5.4%)
Coverage TraceCov
Stdout Web App | Schemathesis

LiteLLM

FastAPI OpenAPI 3.1.0 60k
191,891 requests
1h 0m 0s duration
Proxy server that serves 100+ LLM providers behind one OpenAI-compatible API, with keys, teams and budgets
3,230

Issues Found

873 of 905 API operations affected

See Report
530
Server Errors
5
Security
2,683
Contract Violations
12
Other

Coverage

Timeline
Lines43.0%
56,557 / 131,389
Branches21.8%
10,064 / 46,104
Operations99.9%
904 / 905
Parameters79.6%
975 / 1,225
Keywords65.8%
15,958 / 24,264

API Responses

25,000
2xx (13.0%)
146,080
4xx (76.2%)
20,584
5xx (10.7%)
Coverage Code | TraceCov
Stdout Web App | Schemathesis

Gitea

chi OpenAPI 2.0 58k
174,058 requests
1h 0m 0s duration
Self-hosted Git service with issues, pull requests, packages and CI
1,074

Issues Found

423 of 537 API operations affected

See Report
32
Server Errors
84
Security
837
Contract Violations
121
Other

Coverage

Timeline
Operations100.0%
537 / 537
Parameters100.0%
1,677 / 1,677
Keywords100.0%
3,106 / 3,106

API Responses

52,110
2xx (29.9%)
120,972
4xx (69.5%)
971
5xx (0.6%)
Coverage TraceCov
Stdout Web App | Schemathesis

Jellyfin

ASP.NET Core OpenAPI 3.0.4 58k
326,220 requests
1h 0m 2s duration
Self-hosted media server for movies, music and live TV
933

Issues Found

310 of 364 API operations affected

See Report
60
Server Errors
22
Security
846
Contract Violations
5
Other

Coverage

Timeline
Operations99.2%
361 / 364
Parameters95.3%
2,015 / 2,115
Keywords47.4%
13,361 / 28,167

API Responses

70,666
2xx (21.7%)
231,637
4xx (71.0%)
23,917
5xx (7.3%)
Coverage TraceCov
Stdout Web App | Schemathesis

Apache Airflow

FastAPI OpenAPI 3.1 47k
140,951 requests
1h 0m 0s duration
Platform to programmatically author, schedule and monitor workflows
311

Issues Found

89 of 128 API operations affected

See Report
41
Server Errors
267
Contract Violations
3
Other

Coverage

Timeline
Lines73.6%
8,461 / 11,498
Branches43.2%
1,230 / 2,844
Operations97.7%
125 / 128
Parameters90.7%
566 / 624
Keywords96.2%
2,353 / 2,447

API Responses

37,967
2xx (27.0%)
101,534
4xx (72.3%)
912
5xx (0.6%)
Coverage Code | TraceCov
Stdout Web App | Schemathesis

Paperless-ngx

Django REST Framework OpenAPI 3.0.3 46k
273,690 requests
1h 0m 0s duration
Document management system that scans, indexes and archives paper documents
719

Issues Found

161 of 163 API operations affected

See Report
79
Server Errors
1
Security
639
Contract Violations

Coverage

Timeline
Lines47.2%
8,844 / 18,742
Branches21.4%
1,095 / 5,106
Operations99.4%
162 / 163
Parameters84.1%
503 / 598
Keywords75.8%
7,155 / 9,439

API Responses

75,401
2xx (27.6%)
185,888
4xx (67.9%)
12,381
5xx (4.5%)
Coverage Code | TraceCov
Stdout Web App | Schemathesis

Qdrant

Actix Web OpenAPI 3.0.1 35k
130,960 requests
20m 28s duration
Vector similarity search engine and vector database
120

Issues Found

57 of 69 API operations affected

See Report
11
Server Errors
109
Contract Violations

Coverage

Timeline
Operations100.0%
69 / 69
Parameters80.6%
154 / 191
Keywords66.7%
5,937 / 8,898

API Responses

9,986
2xx (7.6%)
112,641
4xx (86.0%)
8,325
5xx (6.4%)
Coverage TraceCov
Stdout Web App | Schemathesis

PostgREST

Warp OpenAPI 2.0 28k
528,821 requests
1h 0m 0s duration
REST API generated from a PostgreSQL schema, serving a small library catalog
202

Issues Found

26 of 26 API operations affected

See Report
6
Server Errors
10
Security
186
Contract Violations

Coverage

Timeline
Operations96.3%
26 / 27
Parameters92.5%
149 / 161
Keywords89.4%
294 / 329

API Responses

185,829
2xx (35.2%)
342,625
4xx (64.8%)
40
5xx (0.0%)
Coverage TraceCov
Stdout Web App | Schemathesis

Firefly III

Laravel OpenAPI 3.0.0 25k
107,332 requests
1h 0m 0s duration
Personal finance manager with accounts, budgets, bills and rules
801

Issues Found

243 of 243 API operations affected

See Report
57
Server Errors
45
Security
699
Contract Violations

Coverage

Timeline
Operations100.0%
243 / 243
Parameters80.4%
686 / 853
Keywords61.3%
2,528 / 4,122

API Responses

62,420
2xx (58.2%)
43,245
4xx (40.3%)
1,661
5xx (1.5%)
Coverage TraceCov
Stdout Web App | Schemathesis

Prefect

FastAPI OpenAPI 3.1 24k
27,308 requests
1h 0m 0s duration
Workflow orchestration framework for building resilient data pipelines in Python
506

Issues Found

170 of 187 API operations affected

See Report
55
Server Errors
449
Contract Violations
2
Other

Coverage

Timeline
Lines67.1%
14,648 / 21,820
Branches36.5%
1,994 / 5,468
Operations100.0%
187 / 187
Parameters87.2%
361 / 414
Keywords47.4%
7,985 / 16,852

API Responses

7,009
2xx (25.7%)
19,737
4xx (72.3%)
561
5xx (2.1%)
Coverage Code | TraceCov
Stdout Web App | Schemathesis

NetBox

Django REST Framework OpenAPI 3.0.3 22k
65,721 requests
1h 0m 0s duration
Network source of truth for modelling and documenting infrastructure
2,175

Issues Found

1,222 of 1,255 API operations affected

See Report
54
Server Errors
201
Security
1,920
Contract Violations

Coverage

Timeline
Lines66.7%
39,865 / 59,785
Branches14.8%
1,638 / 11,042
Operations99.9%
1,255 / 1,256
Parameters48.8%
9,309 / 19,087
Keywords21.7%
33,522 / 154,759

API Responses

10,272
2xx (15.6%)
53,751
4xx (81.8%)
1,698
5xx (2.6%)
Coverage Code | TraceCov
Stdout Web App | Schemathesis

OpenProject

Rails OpenAPI 3.1.2 16k
350,433 requests
1h 0m 0s duration
Project management with work packages, boards, time tracking and meetings
1,199

Issues Found

317 of 320 API operations affected

See Report
115
Server Errors
1
Security
1,083
Contract Violations

Coverage

Timeline
Operations100.0%
320 / 320
Parameters94.8%
454 / 479
Keywords92.5%
2,429 / 2,627

API Responses

63,980
2xx (18.3%)
266,149
4xx (76.0%)
20,203
5xx (5.8%)
Coverage TraceCov
Stdout Web App | Schemathesis

Unleash

Express OpenAPI 3.0.3 14k
211,716 requests
24m 27s duration
Feature flag management with projects, environments, strategies and segments
495

Issues Found

156 of 211 API operations affected

See Report
58
Server Errors
3
Security
427
Contract Violations
7
Other

Coverage

Timeline
Operations100.0%
211 / 211
Parameters99.4%
333 / 335
Keywords98.8%
2,234 / 2,260

API Responses

94,303
2xx (44.5%)
104,683
4xx (49.4%)
12,722
5xx (6.0%)
Coverage TraceCov
Stdout Web App | Schemathesis

Mealie

FastAPI OpenAPI 3.0 13k
119,173 requests
1h 0m 0s duration
Self-hosted recipe manager and meal planner with a RestAPI backend
768

Issues Found

232 of 247 API operations affected

See Report
103
Server Errors
665
Contract Violations

Coverage

Timeline
Lines74.6%
13,064 / 17,510
Branches45.1%
1,566 / 3,476
Operations96.4%
238 / 247
Parameters91.9%
684 / 744
Keywords91.5%
5,079 / 5,553

API Responses

47,890
2xx (40.2%)
60,683
4xx (50.9%)
10,587
5xx (8.9%)
Coverage Code | TraceCov
Stdout Web App | Schemathesis

OpenReplay

FastAPI OpenAPI 3.1.0 13k
204,322 requests
1h 1m 36s duration
Session replay and product analytics platform
484

Issues Found

142 of 147 API operations affected

See Report
72
Server Errors
411
Contract Violations
1
Other

Coverage

Timeline
Lines51.5%
4,177 / 8,109
Branches22.4%
543 / 2,428
Operations98.0%
144 / 147
Parameters98.9%
179 / 181
Keywords98.4%
762 / 774

API Responses

46,018
2xx (22.5%)
149,901
4xx (73.4%)
8,333
5xx (4.1%)
Coverage Code | TraceCov
Stdout Web App | Schemathesis

Polar

FastAPI OpenAPI 3.1 10k
81,862 requests
1h 0m 0s duration
Turn your software into a business - payment and monetization platform for open source
601

Issues Found

211 of 229 API operations affected

See Report
89
Server Errors
5
Security
506
Contract Violations
1
Other

Coverage

Timeline
Lines65.7%
19,848 / 30,218
Branches30.0%
1,882 / 6,282
Operations96.9%
222 / 229
Parameters58.4%
298 / 510
Keywords83.8%
4,580 / 5,465

API Responses

21,359
2xx (26.3%)
57,881
4xx (71.2%)
2,055
5xx (2.5%)
Coverage Code | TraceCov
Stdout Web App | Schemathesis

InvenTree

Django REST Framework OpenAPI 3.0 7.7k
34,928 requests
1h 0m 0s duration
Open source inventory management system with a Django REST Framework API
1,327

Issues Found

551 of 583 API operations affected

See Report
100
Server Errors
7
Security
1,220
Contract Violations

Coverage

Timeline
Lines59.2%
22,295 / 37,680
Branches23.2%
2,250 / 9,692
Operations98.6%
575 / 583
Parameters45.1%
991 / 2,197
Keywords36.7%
6,089 / 16,598

API Responses

8,563
2xx (24.5%)
24,741
4xx (70.9%)
1,615
5xx (4.6%)
Coverage Code | TraceCov
Stdout Web App | Schemathesis

Weblate

Django REST Framework OpenAPI 3.1 6.1k
148,506 requests
1h 0m 0s duration
Web-based continuous localization system with a Django REST Framework API
338

Issues Found

123 of 160 API operations affected

See Report
18
Server Errors
302
Contract Violations
18
Other

Coverage

Timeline
Lines47.8%
23,212 / 48,512
Branches18.3%
2,597 / 14,168
Operations90.1%
145 / 161
Parameters82.9%
495 / 597
Keywords75.9%
4,481 / 5,905

API Responses

40,466
2xx (27.3%)
106,898
4xx (72.2%)
794
5xx (0.5%)
Coverage Code | TraceCov
Stdout Web App | Schemathesis

OPAL

FastAPI OpenAPI 3.1.0 5.5k
348,439 requests
1h 0m 0s duration
Administration layer that keeps policy agents such as OPA and Cedar up to date in real time
70

Issues Found

17 of 20 API operations affected

See Report
12
Server Errors
55
Contract Violations
3
Other

Coverage

Timeline
Lines56.7%
3,312 / 5,845
Branches28.6%
348 / 1,218
Operations100.0%
20 / 20
Parameters97.3%
36 / 37
Keywords99.6%
256 / 257

API Responses

145,506
2xx (41.8%)
99,709
4xx (28.6%)
103,158
5xx (29.6%)
Coverage Code | TraceCov
Stdout Web App | Schemathesis

Dependency-Track

Jersey OpenAPI 3.0.1 4.3k
581,201 requests
1h 0m 0s duration
Software composition analysis platform that tracks components and vulnerabilities from SBOMs
532

Issues Found

197 of 219 API operations affected

See Report
101
Server Errors
3
Security
423
Contract Violations
5
Other

Coverage

Timeline
Operations100.0%
219 / 219
Parameters92.3%
550 / 596
Keywords9.1%
1,012 / 11,101

API Responses

136,055
2xx (23.4%)
421,447
4xx (72.5%)
23,578
5xx (4.1%)
Coverage TraceCov
Stdout Web App | Schemathesis

Kuma

go-restful OpenAPI 3.1.0 4.0k
431,839 requests
1h 0m 0s duration
Service mesh control plane for Kubernetes and VMs
357

Issues Found

139 of 141 API operations affected

See Report
357
Contract Violations

Coverage

Timeline
Operations100.0%
141 / 141
Parameters98.9%
363 / 367
Keywords99.3%
3,561 / 3,585

API Responses

99,808
2xx (23.1%)
331,751
4xx (76.9%)
0
5xx (0.0%)
Coverage TraceCov
Stdout Web App | Schemathesis

pygeoapi

Flask OpenAPI 3.0 626
175,347 requests
1h 0m 0s duration
OGC API server for geospatial data, built on Flask
189

Issues Found

59 of 66 API operations affected

See Report
14
Server Errors
175
Contract Violations

Coverage

Timeline
Lines61.0%
3,412 / 5,590
Branches46.2%
769 / 1,664
Operations98.5%
66 / 67
Parameters88.8%
190 / 214
Keywords93.3%
376 / 403

API Responses

78,299
2xx (44.7%)
94,263
4xx (53.8%)
2,642
5xx (1.5%)
Coverage Code | TraceCov
Stdout Web App | Schemathesis

Openverse

Django REST Framework OpenAPI 3.0 380
172,394 requests
1h 0m 0s duration
Search engine API for openly licensed images and audio
29

Issues Found

12 of 17 API operations affected

See Report
2
Server Errors
27
Contract Violations

Coverage

Timeline
Lines78.3%
2,977 / 3,801
Branches51.4%
323 / 628
Operations100.0%
17 / 17
Parameters100.0%
70 / 70
Keywords100.0%
240 / 240

API Responses

23,876
2xx (13.9%)
148,419
4xx (86.1%)
33
5xx (0.0%)
Coverage Code | TraceCov
Stdout Web App | Schemathesis

Covid 19 Japan Web API

Flask OpenAPI 2.0 129
25,207 requests
2m 0s duration
Web API to get COVID-19 information of each prefecture in Japan
1

Issue Found

1 of 4 API operations affected

See Report
1
Contract Violations

Coverage

Timeline
Lines95.2%
80 / 84
Branches87.5%
7 / 8
Operations100.0%
4 / 4
Parameters100.0%
3 / 3
Keywords100.0%
3 / 3

API Responses

12,775
2xx (50.7%)
12,432
4xx (49.3%)
0
5xx (0.0%)
Coverage Code | TraceCov
Stdout Web App | Schemathesis

aeman

net/http OpenAPI 3.0.3 34
554,074 requests
1h 0m 0s duration
Sprint and planning board for engineering teams, stored as commits in a git repository
120

Issues Found

51 of 60 API operations affected

See Report
12
Server Errors
108
Contract Violations

Coverage

Timeline
Operations98.4%
60 / 61
Parameters100.0%
164 / 164
Keywords100.0%
361 / 361

API Responses

181,699
2xx (32.8%)
350,154
4xx (63.2%)
22,135
5xx (4.0%)
Coverage TraceCov
Stdout Web App | Schemathesis

REST-ler Blog API

Flask OpenAPI 2.0
5,965 requests
5m 0s duration
Demo API used in the original REST-ler paper
21

Issues Found

5 of 6 API operations affected

See Report
2
Server Errors
19
Contract Violations

Coverage

Timeline
Lines94.0%
173 / 184
Branches66.7%
12 / 18
Operations100.0%
6 / 6
Parameters100.0%
5 / 5
Keywords100.0%
6 / 6

API Responses

4,180
2xx (70.1%)
1,648
4xx (27.6%)
134
5xx (2.2%)
Coverage Code | TraceCov
Stdout Web App | Schemathesis

Sample API

FastAPI OpenAPI 3.1.0
1,918 requests
1m 0s duration
A sample API designed to test the coverage effectiveness of Schemathesis tests
0

Issues Found

0 of 1 API operations affected

See Report

Coverage

Timeline
Lines100.0%
5 / 5
Operations100.0%
1 / 1

API Responses

1,913
2xx (99.7%)
5
4xx (0.3%)
0
5xx (0.0%)
Coverage Code | TraceCov
Stdout Web App | Schemathesis

Ready to test your API?

Start finding bugs in minutes with a single command

uvx schemathesis run <YOUR API SPEC URL>
Get Started Documentation

Want the same schema coverage for your own tests? Try TraceCov