https://gitlab.synchro.net/main/sbbs/-/commit/99cd3ea465ae8306418e1eeb
Added Files:
ctrl/web_block.ini src/sbbs3/web_block.hpp
Modified Files:
docs/v322_new.md src/sbbs3/websrvr.cpp
Log Message:
Web Server: block requests by request-header pattern (ctrl/web_block.ini)
Scrapers are often identifiable only by a request header, most often the User-Agent, and with HTTPS a packet filter cannot see headers. The new ctrl/web_block.ini has one [header:<name>] section per header to check;
its numbered keys are patterns in the findstr syntax of the .can/.cfg
files, tried in numeric key order. A request whose header matches is
refused with "403 Forbidden" and a log line naming the header and the
pattern.
The check runs once the request headers are parsed, before webctrl.ini, authentication or any script. Running after parse_headers() means the
client address is the one from RemoteIPHeader when the server is behind
a reverse proxy, so the proxy itself is neither exempted nor filtered.
The 403 skips any SSJS error page (a static 403.html is still sent), so
a blocked request never starts a JavaScript context.
A ! pattern is an exception, so a section that ends in ! patterns is an allowlist. [header:host] is matched against the host name the server
uses for the request (lower-case, no :port, taken from the request line
of an absolute-form request), or the server's own host name when the
request has no Host header, so "0 = !www.example.com" serves one host
name and refuses every other. A header the request does not send
matches no pattern. Only the first section for a header is used; a
duplicate is reported when the file is loaded.
A section's FilterThreshold (default 0, never) adds a client to ip.can
once that many of its requests have been blocked by that header. Clients
are counted like the rate limiter's: by IP address, or by subnet per RateLimitSubnetPrefix4/6, filtering the whole subnet only once RateLimitFilterSubnetThreshold of its addresses have been blocked. A
count is forgotten after an hour with no blocked request. FilterDuration
and FilterSilent default to the [Web] RateLimitFilterDuration and RateLimitFilterSilent settings. Clients in ipfilter_exempt.cfg are never blocked.
The file is re-read when its date changes (checked at most every cache_filter_files seconds), so edits take effect without a recycle. A
file that cannot be opened keeps the rules already loaded and is retried
at the next check, and a file changed within the last second is read
again, in case it was read while still being written. Like
web_alias.ini and web_handler.ini, it may be a per-host
(web_block.<host>.ini) or .local variant. The stock file holds only
commented documentation and examples.
Tested on Linux against an isolated test server: User-Agent and Referer patterns (contains, begins-with, case), ! exceptions, the Host allowlist
with a :port, a missing Host, and absolute-form requests; the threshold
filter, its duration, per-header counts and subnet filtering; requests
through a proxy (RemoteIPHeader) from an exempt address; the static 403
page with a 403.ssjs present; a duplicate section; edits, removal,
restoration and an unreadable file; an exempt client; a per-host file
name.
Fixes #1375
Co-Authored-By: Claude Opus 5.5 <
noreply@anthropic.com>
---
■ Synchronet ■ Vertrauen ■ Home of Synchronet ■ [vert/cvs/bbs].synchro.net