Sentinel ML classifier: evaluation
Generated by go run ./cmd/mltrain (2026-11-18). Model: logistic regression over hashed character 3-5-grams, words and word pairs (32768 slots), trained on 99118 examples (half attacks from 24 generated families and public payload lists, half normal input, generated and recorded from real sites). Threshold: 0.90.
Most of the data is generated. The numbers show how the model behaves on fresh data from the same generator, on evasion variants and public payloads it never saw, and on real browser traffic from sites it never saw; your traffic will differ. Run Sentinel with
ML_MODE=detecton real traffic first and check the log before blocking.
1. Fresh test data (5000 attacks, 5000 normal, different seed)
| Attacks caught | Normal input wrongly blocked | |
|---|---|---|
| WAF rules | 74.6% (3730/5000) | 0.0% (0/5000) |
| ML | 99.5% (4975/5000) | 0.0% (0/5000) |
| WAF + ML | 99.8% (4989/5000) | 0.0% (0/5000) |
Attacks caught per family
| Family | WAF rules | ML | WAF + ML |
|---|---|---|---|
| cmdi | 81.8% (189/231) | 93.5% (216/231) | 98.7% (228/231) |
| crlf | 100.0% (189/189) | 100.0% (189/189) | 100.0% (189/189) |
| file-uri | 47.5% (94/198) | 100.0% (198/198) | 100.0% (198/198) |
| jndi | 100.0% (211/211) | 100.0% (211/211) | 100.0% (211/211) |
| ldap | 66.5% (159/239) | 100.0% (239/239) | 100.0% (239/239) |
| nosqli | 37.7% (77/204) | 100.0% (204/204) | 100.0% (204/204) |
| sqli-blind | 6.3% (13/206) | 100.0% (206/206) | 100.0% (206/206) |
| sqli-error | 23.2% (42/181) | 100.0% (181/181) | 100.0% (181/181) |
| sqli-orderby | 0.0% (0/229) | 100.0% (229/229) | 100.0% (229/229) |
| sqli-schema | 97.1% (203/209) | 100.0% (209/209) | 100.0% (209/209) |
| sqli-sqlmap | 71.6% (144/201) | 98.0% (197/201) | 99.0% (199/201) |
| sqli-stacked | 100.0% (222/222) | 100.0% (222/222) | 100.0% (222/222) |
| sqli-tautology | 35.5% (77/217) | 97.7% (212/217) | 97.7% (212/217) |
| sqli-time | 96.5% (191/198) | 100.0% (198/198) | 100.0% (198/198) |
| sqli-union | 91.4% (181/198) | 100.0% (198/198) | 100.0% (198/198) |
| sqli-waitfor | 100.0% (211/211) | 100.0% (211/211) | 100.0% (211/211) |
| ssrf | 76.7% (165/215) | 100.0% (215/215) | 100.0% (215/215) |
| ssti | 88.4% (191/216) | 100.0% (216/216) | 100.0% (216/216) |
| traversal | 97.9% (186/190) | 100.0% (190/190) | 100.0% (190/190) |
| xss-event | 94.1% (174/185) | 100.0% (185/185) | 100.0% (185/185) |
| xss-script | 100.0% (174/174) | 100.0% (174/174) | 100.0% (174/174) |
| xss-template | 100.0% (234/234) | 100.0% (234/234) | 100.0% (234/234) |
| xss-uri | 83.7% (200/239) | 99.6% (238/239) | 99.6% (238/239) |
| xxe | 100.0% (203/203) | 100.0% (203/203) | 100.0% (203/203) |
Normal input wrongly blocked, per kind
| Kind | WAF rules | ML |
|---|---|---|
| address | 0.0% (0/182) | 0.0% (0/182) |
| arabic | 0.0% (0/185) | 0.0% (0/185) |
| code-ish | 0.0% (0/179) | 0.0% (0/179) |
| cookie-value | 0.0% (0/181) | 0.0% (0/181) |
| date | 0.0% (0/193) | 0.0% (0/193) |
| 0.0% (0/206) | 0.0% (0/206) | |
| filename | 0.0% (0/215) | 0.0% (0/215) |
| graphql | 0.0% (0/175) | 0.0% (0/175) |
| hard-negative | 0.0% (0/153) | 0.0% (0/153) |
| header-value | 0.0% (0/192) | 0.0% (0/192) |
| id | 0.0% (0/184) | 0.0% (0/184) |
| json-text | 0.0% (0/183) | 0.0% (0/183) |
| name | 0.0% (0/167) | 0.0% (0/167) |
| number | 0.0% (0/175) | 0.0% (0/175) |
| parens | 0.0% (0/178) | 0.0% (0/178) |
| password | 0.0% (0/177) | 0.0% (0/177) |
| payment | 0.0% (0/171) | 0.0% (0/171) |
| phone | 0.0% (0/176) | 0.0% (0/176) |
| review | 0.0% (0/154) | 0.0% (0/154) |
| search | 0.0% (0/172) | 0.0% (0/172) |
| sentence | 0.0% (0/164) | 0.0% (0/164) |
| socketio | 0.0% (0/196) | 0.0% (0/196) |
| tech-prose | 0.0% (0/172) | 0.0% (0/172) |
| telemetry | 0.0% (0/184) | 0.0% (0/184) |
| token | 0.0% (0/166) | 0.0% (0/166) |
| url | 0.0% (0/162) | 0.0% (0/162) |
| user-agent | 0.0% (0/184) | 0.0% (0/184) |
| words | 0.0% (0/174) | 0.0% (0/174) |
Public payloads the model never saw (1153, PayloadsAllTheThings)
One in five payloads from each list was held out of training. Sent as a search parameter.
| Family | Payloads | WAF rules | ML | WAF + ML |
|---|---|---|---|---|
| all | 1153 | 55.2% (637/1153) | 94.5% (1090/1153) | 94.8% (1093/1153) |
| public-cmdi | 95 | 40.0% (38/95) | 92.6% (88/95) | 93.7% (89/95) |
| public-ldap | 6 | 16.7% (1/6) | 33.3% (2/6) | 33.3% (2/6) |
| public-nosql | 5 | 60.0% (3/5) | 80.0% (4/5) | 80.0% (4/5) |
| public-sqli | 248 | 77.4% (192/248) | 96.0% (238/248) | 96.4% (239/248) |
| public-ssrf | 150 | 0.0% (0/150) | 92.7% (139/150) | 92.7% (139/150) |
| public-ssti | 25 | 28.0% (7/25) | 88.0% (22/25) | 88.0% (22/25) |
| public-traversal | 308 | 65.6% (202/308) | 99.7% (307/308) | 99.7% (307/308) |
| public-xss | 294 | 61.9% (182/294) | 91.2% (268/294) | 91.5% (269/294) |
| public-xxe | 22 | 54.5% (12/22) | 100.0% (22/22) | 100.0% (22/22) |
Real browser traffic the model never saw (open-appsec dataset)
Normal requests recorded while browsing real sites. Sessions on half of the sites were used for training; these 85207 requests from the other 318 sites (whole sites, not just sessions) were not. Every value the gateway would score (query and body fields, headers) was scored.
| Model | Requests | Wrongly flagged by ML |
|---|---|---|
| before this training | 85207 | 3.0% (2581/85207) |
| this model | 85207 | 0.7% (611/85207) |
Examples still flagged:
entries[].server_ip=127.0.0.1:8888entries[].server_ip=127.0.0.1:8888entries[].server_ip=127.0.0.1:8888entries[].server_ip=127.0.0.1:8888entries[].server_ip=127.0.0.1:8888[].event_data.method=onSubmit[].event_data.method=onFocus[].event_data.method=onFocus
2. Evasion test (4000 attacks, each rewritten with 1-3 evasion techniques)
| Technique | Attacks | WAF rules | ML | WAF + ML |
|---|---|---|---|---|
| all | 4000 | 72.0% (2878/4000) | 99.2% (3970/4000) | 99.7% (3987/4000) |
| HTML entities | 787 | 69.9% (550/787) | 99.9% (786/787) | 99.9% (786/787) |
| MySQL version comments | 800 | 70.9% (567/800) | 98.8% (790/800) | 99.8% (798/800) |
| URL encoding | 769 | 72.6% (558/769) | 99.6% (766/769) | 99.9% (768/769) |
| XSS tag tricks | 787 | 69.4% (546/787) | 98.7% (777/787) | 99.4% (782/787) |
| double URL encoding | 826 | 74.7% (617/826) | 99.9% (825/826) | 100.0% (826/826) |
| no spaces (parentheses, IFS) | 799 | 71.6% (572/799) | 99.4% (794/799) | 99.7% (797/799) |
| operator swaps | 797 | 67.3% (536/797) | 98.7% (787/797) | 99.2% (791/797) |
| random case | 857 | 72.1% (618/857) | 98.9% (848/857) | 99.5% (853/857) |
| shell quote splitting | 812 | 70.1% (569/812) | 98.9% (803/812) | 99.3% (806/812) |
| tabs and newlines for spaces | 772 | 72.7% (561/772) | 98.8% (763/772) | 99.6% (769/772) |
Examples both missed (ideas for new rules and training data):
t-shirt+Or 2967=2967+#Rock 'n' roll t-shirt, size L file:///e''tc/pa''sswd Is there a discount for orders over $100?I'm/**/not/**/sure/**/if/**/2/**/>/**/1/**/applies/**/here/**/@(7*7)/**/How/**/do/**/I/**/update/**/my/**/address?WE+MEt+AT+UNION+sqUare+At+5pm.+%2BOrdEr+by+14%2b+iS+tHere+a+DIsCOUnT+for+ORDErS+oveR+%24100%3Ftrue,/**/$/*!50000where*/:/**/'1/**/ like like /**/1'{"uSerNAmE&QuOT;: {&QuoT;$iN&QuOt;: [&QUot;Admin&qUot;,&QuOt;rOoT&qUot;]}}- `Does this come in blue?