1. Forum
  2. FSXNet
  3. FSX MYS

  • Python parsing of Pascal records

    From Analog@21:2/123 to All on Mon Dec 30 00:13:06 2019
    I'm working on a lightbar file viewer, let's just call it that for now. I'm trying to parse FBASES.DAT which is the pascal database for Mystic. Problem
    I'm having is with types. I have the record structure from RECORDS.112 (in
    Doc folder). However, as I parse over the FBASES file, it appears there are extra bytes, not following the record shape.

    I think I saw somewhere about the first byte of a record entry is the length
    of the entry. Does this sound right? For instance, if I need to read a String of 30 bytes (chars) from a pascal record, do I first need to read byte 1 to
    get the true length of the String?

    Anyhow, almost got this thing licked.

    |19|15┌─|16|07┤ |08De|07ad|15be|07a|08tz b|07b|15s
    |07└─┘├─┐ |08:>.|12F|04sx |1221|08:|122|08/|12123|08.
    |11■ |07└|19|15─|16|07┘ |08:>.|10A|02gn |1046|08:|101|08/|10123|08.
    |12≡|15A|07n|08al|07o|15g|12≡ |08:>.|13F|05dn |131|08:|13305|08/|132|08.
    |08:>.|15S|08ci |1577|08:|151|08/|15131|08.
    |08:>.|11T|03qw |111337|08:|113|08/|1113|08.

    --- Mystic BBS v1.12 A43 2019/03/03 (Windows/32)
    * Origin: deadbeatz.org (21:2/123)
  • From xqtr@21:1/111 to Analog on Tue Dec 31 09:45:18 2019
    I think I saw somewhere about the first byte of a record entry is the length of the entry. Does this sound right? For instance, if I need to read a String of 30 bytes (chars) from a pascal record, do I first need
    to read byte 1 to get the true length of the String?

    Yes this is true. If for example the record has a value like String[30], then you have to read the first byte to get the "true" string length ex.20 and then read the rest 30 bytes. So the total size of a string[30] is 31 bytes.

    In python you can use the struct lib. and either use a x type to ignore the first byte or a byte type to read it. If you read NULL mag. there are plenty code snippets for reading MBASES, LASTCALLER, ONELINER etc data files. ;)

    :: XQTR :: Another Droid BBS :: andr01d.zapto.org:9999 :: xqtr@gmx.com

    --- Mystic BBS v1.12 A43 2019/03/03 (Raspberry Pi/32)
    * Origin: Another Droid BBS # andr01d.zapto.org:9999 (21:1/111)
  • From Analog@21:2/123 to xqtr on Tue Dec 31 07:08:47 2019
    Yes thank you. I figured this to be the case. If I skip the first byte and readin the string in, will it be properly terminated or do I need the first byte to know where to terminate the array? I'm using Numpy for my struct to have deterministic byte lengths, e.g., numpy.chararray[30] for string[30].

    |19|15┌─|16|07┤ |08De|07ad|15be|07a|08tz b|07b|15s
    |07└─┘├─┐ |08:>.|12F|04sx |1221|08:|122|08/|12123|08.
    |11■ |07└|19|15─|16|07┘ |08:>.|10A|02gn |1046|08:|101|08/|10123|08.
    |12≡|15A|07n|08al|07o|15g|12≡ |08:>.|13F|05dn |131|08:|13305|08/|132|08.
    |08:>.|15S|08ci |1577|08:|151|08/|15131|08.
    |08:>.|11T|03qw |111337|08:|113|08/|1113|08.

    --- Mystic BBS v1.12 A43 2019/03/03 (Windows/32)
    * Origin: deadbeatz.org (21:2/123)
  • From xqtr@21:1/111 to Analog on Tue Dec 31 14:47:24 2019
    Yes thank you. I figured this to be the case. If I skip the first byte
    and readin the string in, will it be properly terminated or do I need
    the first byte to know where to terminate the array? I'm using Numpy for my struct to have deterministic byte lengths, e.g., numpy.chararray[30] for string[30].

    I think that python needs null terminated strings, which doesn't apply in this case. So it's more easy to read the length first and use it to read the rest
    of the string. If you read the whole 30bytes of the string and print it, it will also print some garbage text. Another way is to skip the length, read the whole string (30bytes) and do a strip function to strip empty space at the end of the string. Whatever works for you :)

    :: XQTR :: Another Droid BBS :: andr01d.zapto.org:9999 :: xqtr@gmx.com

    --- Mystic BBS v1.12 A43 2019/03/03 (Raspberry Pi/32)
    * Origin: Another Droid BBS # andr01d.zapto.org:9999 (21:1/111)
  • From Analog@21:2/123 to xqtr on Tue Dec 31 12:52:54 2019
    Another way is to
    skip the length, read the whole string (30bytes) and do a strip function to strip empty space at the end of the string. Whatever works for you :)

    This worked perfect. I know you use sqlite on linux to index against. I'm afraid with my FileRecord routine, everytime a user clicks on a new filebase, opening the records, reading each entry, then opening the description and seeking to the current selection will be slow. We'll see. I may need to just create a standalone file index database using Python and sqlite3. Maybe just run a nightly event to rebuild it against Mystic's file databases so they are at parity.

    |19|15┌─|16|07┤ |08De|07ad|15be|07a|08tz b|07b|15s
    |07└─┘├─┐ |08:>.|12F|04sx |1221|08:|122|08/|12123|08.
    |11■ |07└|19|15─|16|07┘ |08:>.|10A|02gn |1046|08:|101|08/|10123|08.
    |12≡|15A|07n|08al|07o|15g|12≡ |08:>.|13F|05dn |131|08:|13305|08/|132|08.
    |08:>.|15S|08ci |1577|08:|151|08/|15131|08.
    |08:>.|11T|03qw |111337|08:|113|08/|1113|08.

    --- Mystic BBS v1.12 A43 2019/03/03 (Windows/32)
    * Origin: deadbeatz.org (21:2/123)