• Detecting Terminals

    From dingo@21:1/121 to All on Wed Mar 20 08:31:30 2024
    Hello,

    I was the first to write a BBS that supports UTF-8 and cp437, but we required selecting it manually (x84/default/charset.py script). But I wanted to share with you a technique that I've been using lately that should allow your BBS Softwares to automatically detect the client, using the "Query Cursor Position" sequence, "\x1b[6n". You could also use this to automatically detect for "bots" or "scanners" rather than real users. At least, any bot would have to emulate a terminal to respond correctly.

    I use this technique to automatically detect the unicode support level of many terminals in my "ucs-detect" tool, "How it works" https://ucs-detect.readthedocs.io/intro.html#how-it-works

    And the results here, https://ucs-detect.readthedocs.io/results.html

    In any case, if the "bots" are not responding to this query, you can kick them off. You can detect if they support utf-8, and, whether they support emojis etc. CP437 terminals like SyncTerm will be "corrupted" by utf-8 bytes or display characters incorrectly, and by measuring the delta of the starting and ending position of a character after displaying unicode string, we can determine whether they are utf-8 capable automatically.

    I could write such a tool if anyone is interested, it would be very easy. Anyway I more highly suggest to use this to kick off "bots", as they can really consume a lot of resources.





    --- ENiGMA 1/2 v0.0.14-beta (linux; x64; 18.18.2)
    * Origin: Xibalba -+- xibalba.l33t.codes:44510 (21:1/121)
  • From dingo@21:1/121 to dingo on Wed Mar 20 08:35:20 2024

    On Wednesday, March 20th dingo muttered...
    I use this technique to automatically detect the unicode support level of many terminals in my "ucs-detect" tool, "How it works" https://ucs-detect.readthedocs.io/intro.html#how-it-works

    I also wanted to say, and I may still make this project someday, by storing database records of terminal unicode support levels and introspection of telnet client capabilities (just blast all known telnet negotiations and record their responses), it would be trivial to identify *exactly* what terminal emulator or client is used. I could say exactly "This is SyncTerm/etherterm/xterm/foot/Konsole/iTerm2/Terminal.exe" etc.

    I really wish all terminal capabilities, such as 24-bit color support, sixel, etc., supported a query string to detect whether it is supported. Unfortunately this isn't the case, but we could deduce it with this rather complex code path.



    --- ENiGMA 1/2 v0.0.14-beta (linux; x64; 18.18.2)
    * Origin: Xibalba -+- xibalba.l33t.codes:44510 (21:1/121)
  • From niter3@21:1/199 to dingo on Wed Mar 20 16:11:46 2024
    I was the first to write a BBS that supports UTF-8 and cp437, but we required selecting it manually (x84/default/charset.py script). But I wanted to share with you a technique that I've been using lately that should allow your BBS Softwares to automatically detect the client,
    using the "Query Cursor Position" sequence, "\x1b[6n". You could also
    use this to automatically detect for "bots" or "scanners" rather than
    real users. At least, any bot would have to emulate a terminal to
    respond correctly.

    I use this technique to automatically detect the unicode support level
    of many terminals in my "ucs-detect" tool, "How it works" https://ucs-detect.readthedocs.io/intro.html#how-it-works

    And the results here, https://ucs-detect.readthedocs.io/results.html

    This could come handy. As of right now I'm just hardcoding it to cp437.

    ... "No comment" is a comment.

    --- Mystic BBS v1.12 A49 2023/04/30 (Linux/64)
    * Origin: Clutch BBS * telnet://clutchbbs.com (21:1/199)
  • From AKAcastor@21:1/162 to Dingo on Wed Mar 20 14:51:54 2024
    I was the first to write a BBS that supports UTF-8 and
    cp437, but we required selecting it manually
    (x84/default/charset.py script).

    Supporting UTF-8 is a nice touch, certainly makes it easier for callers to use a variety of modern terminals.

    But I wanted to share
    with you a technique that I've been using lately that
    should allow your BBS Softwares to automatically detect
    the client, using the "Query Cursor Position" sequence,
    "\x1b[6n". You could also use this to automatically
    detect for "bots" or "scanners" rather than real users.
    At least, any bot would have to emulate a terminal to
    respond correctly.

    This same sequence can also be useful for testing for general ANSI support. Nice trick to use it to detect the unicode support level too!

    In any case, if the "bots" are not responding to this
    query, you can kick them off. You can detect if they
    support utf-8, and, whether they support emojis etc.
    CP437 terminals like SyncTerm will be "corrupted" by utf-
    8 bytes or display characters incorrectly, and by
    measuring the delta of the starting and ending position
    of a character after displaying unicode string, we can
    determine whether they are utf-8 capable automatically.

    Great technique and thank you for sharing it. Since my BBS doesn't support UTF-8 (yet) I might use the detection to present a message to callers using UTF-8 terminals to let them know CP437 would be better.

    I think the idea of testing for terminal emulation to detect bots is interesting. It may run the risk of false positives, if you get callers using odd terminals - probably more concern for those running older systems like CBBS or other pure ASCII systems. I wonder how many (if any) bots support the ANSI terminal emulation - I have noticed that some of the scanners are doing telnet protocol negotiation (not just sending raw text for login attempts). Gonna have to run some tests on these bots!

    On a similar topic of detecting terminals - I have recently implemented sixel detection (and am curious if anyone has other/better techniques).

    Sixel detection isn't too difficult, but from my tests different terminals respond to different queries.

    "\x1b[4c" works with iTerm2
    "\x1b[<0c" works with SyncTerm

    Read the response which should end with 'c'. If the response has ;4 in it then that indicates that sixel is supported.

    For sixel detection on my BBS, I send both strings ("\x1b[4c\x1b[<0c") and read until I receive a 'c' (or time out after a second or two). If I find ";4" in the response then I assume sixel support exists.

    I'm still working on understanding how to specify if a terminal should scale the image when displaying (SyncTerm seems to scale automatically, iTerm2 doesn't).


    Chris/akacastor

    --- Maximus 3.01
    * Origin: Another Millennium - Canada - another.tel (21:1/162)
  • From paulie420@21:2/150 to dingo on Thu Mar 21 20:32:00 2024
    I was the first to write a BBS that supports UTF-8 and cp437, but we required selecting it manually (x84/default/charset.py script). But I

    I could write such a tool if anyone is interested, it would be very easy.

    I'd be interested. I want to show users SIXEL graphics only if they are using SyncTERM - I dunno if you could check for that, but I know SyncTERM gives some kind of output like you listed above...

    Following, and pulled your msg for my Mystic BBS info folder - thanks.



    |07p|15AULIE|1142|07o
    |08.........

    --- Mystic BBS v1.12 A48 (Linux/64)
    * Origin: 2o fOr beeRS bbs>>>20ForBeers.com:1337 (21:2/150)