• Message Purging

    From Avon@21:1/101 to g00r00 on Fri Jan 31 20:36:47 2020
    Hi there.

    [snip]

    + MUTIL's MsgPurge now validates that messages are addressed to valid users
    if the base type is Private or the base allows for Private replies within
    the base. Keep in mind, mixed bases (public that allow private replies)
    will take much longer to process than strictly public bases when purging.

    [snip]

    I think this has caught me out, looks like in my first nightly maint I just lost a ton of message from my netmail bases

    [snip]

    + Jan 31 00:45:06 Process: Purging Message Bases
    + Jan 31 00:45:06 E-mail
    + Jan 31 00:45:06 Purge private messages
    ! Jan 31 00:45:06 Purge #1 Unknown user: SYSOP NAME HERE
    ! Jan 31 00:45:06 Purge #145 Unknown user: TWINK
    ! Jan 31 00:45:06 Purge #146 Unknown user: TWINK
    ! Jan 31 00:45:06 Purge #154 Unknown user: TWINK
    ! Jan 31 00:45:06 Purge #184 Unknown user: VARSUCHI

    [snip]

    ! Jan 31 00:45:10 Purge #11205 Unknown user: LEM0NLIME
    + Jan 31 00:45:10 Purged 39

    [snip]

    + Jan 31 00:45:10 Netmail (FidoNet)
    + Jan 31 00:45:10 Purge private messages
    ! Jan 31 00:45:10 Purge #2 Unknown user: ANDREW CLARKE
    ! Jan 31 00:45:10 Purge #8 Unknown user: AREAFIX
    ! Jan 31 00:45:10 Purge #11 Unknown user: SBBSECHO
    ! Jan 31 00:45:10 Purge #12 Unknown user: SBBSECHO
    ! Jan 31 00:45:10 Purge #16 Unknown user: SBBSECHO
    ! Jan 31 00:45:10 Purge #18 Unknown user: SBBSECHO
    ! Jan 31 00:45:10 Purge #21 Unknown user: DENNIS COLLINS

    [snip]

    ! Jan 31 00:45:11 Purge #1928 Unknown user: DWIGHT SPENCER
    ! Jan 31 00:45:11 Purge #1930 Unknown user: TIM BLADA
    + Jan 31 00:45:11 Purged 548

    [snip]

    and so it goes for my Netmail bases across othernets :(

    + Jan 31 00:45:14 E-mail
    + Jan 31 00:45:15 Removed 41 msgs

    + Jan 31 00:48:22 Netmail (Agoranet)
    + Jan 31 00:48:23 Removed 63 msgs

    + Jan 31 00:48:50 Netmail (FidoNet)
    + Jan 31 00:48:51 Removed 548 msgs

    I'm probably going to have to roll back to A43 tomorrow after I check your reply to this one. Just need to understand this feature better and suss how
    to avoid losing stuff I want to retain.

    Netmail bases impacted had the following set

    Max Msgs │ 0
    Max Msg Age │ 0
    New Scan │ Yes
    QWK Scan │ Yes
    Header │ msghead
    R Template │ ansimrd
    L Template │ ansimlst
    Real Names │ Yes
    Autosigs │ No
    Private │ Yes
    Pvt Reply │ No
    Allow ANSI │ No

    Looks like I need to remove the Private flag right?

    Can you please explain Private and Private Reply for me :)

    Best, Paul

    --- Mystic BBS v1.12 A44 2020/01/29 (Windows/32)
    * Origin: Agency BBS | Dunedin, New Zealand | agency.bbs.nz (21:1/101)
  • From g00r00@21:1/108 to Avon on Fri Jan 31 11:14:50 2020
    + MUTIL's MsgPurge now validates that messages are addressed to valid users if the base type is Private or the base allows for Private replies within the base. Keep in mind, mixed bases (public that allow private

    and so it goes for my Netmail bases across othernets :(

    ! Jan 31 00:45:10 Purge #2 Unknown user: ANDREW CLARKE
    ! Jan 31 00:45:10 Purge #8 Unknown user: AREAFIX
    ! Jan 31 00:45:10 Purge #11 Unknown user: SBBSECHO

    This was something you had asked me to add. It should be removing private messages that are addressed "To" users who aren't members of your BBS. It seems like that is what its doing that.

    What I am thinking is that we may also want to check the From field or sent flags?

    I'm probably going to have to roll back to A43 tomorrow after I check
    your reply to this one. Just need to understand this feature better and suss how to avoid losing stuff I want to retain.

    You could not use the MsgPurge feature that is removing them. Its not really something that needs to be ran a lot anyway, since its goal is to purge old and invalid messages.

    Looks like I need to remove the Private flag right?
    Can you please explain Private and Private Reply for me :)

    Private messages are messages that no one can see except the person they are addressed "To". Netmail falls into that same category. Its removing private messages addressed "To" invalid users because they are messages no one will ever see.

    Reply to Private flag just means that you're allowing users to send a private reply within a public message base. I don't recommend using that personally, but its a feature from the "QuickBBS" style of BBSes and some had asked for it over the years.

    I am going to have it check the From field too though, because I think that is relevant.

    --- Mystic BBS v1.12 A44 2020/01/29 (Linux/64)
    * Origin: Sector 7 (21:1/108)
  • From Avon@21:1/101 to g00r00 on Sat Feb 1 05:59:58 2020
    On 31 Jan 2020 at 11:14a, g00r00 pondered and said...

    This was something you had asked me to add. It should be removing
    private messages that are addressed "To" users who aren't members of
    your BBS. It seems like that is what its doing that.

    What I am thinking is that we may also want to check the From field or sent flags?

    Oh OK, golly.. my memory fails me on that one. I'm not sure how the From and Sent would help? I'm picking if I sent a netmail so I am a known user then
    the message would not be purged even if the To field was to someone outside
    the BBS - am I reading you right?

    It does seem a bit strange to me still that this purge process would be
    applied to Netmail style bases given the nature of Netmail being often to be one of sending to others who may not be a member of your own BBS. Perhaps better to exclude netmail bases from this functions focus?

    You could not use the MsgPurge feature that is removing them. Its not really something that needs to be ran a lot anyway, since its goal is to purge old and invalid messages.

    I use it daily to maintain Usenet bases at a max number of messages... so
    it's not really an option for me to cease this.

    Private messages are messages that no one can see except the person they are addressed "To". Netmail falls into that same category. Its

    So if this were an echomail base and I compose a private message to a User
    (who needs to be a member of my BBS) then that user would see the echomail
    post but no one else? Hmm... nope that can't be it as the message would be exported out.. thinking as I type :)

    are addressed "To". Netmail falls into that same category. Its
    removing private messages addressed "To" invalid users because they are messages no one will ever see.

    This I do get :)

    Reply to Private flag just means that you're allowing users to send a private reply within a public message base. I don't recommend using
    that personally, but its a feature from the "QuickBBS" style of BBSes
    and some had asked for it over the years.

    Private reply how? Sorry having a blond moment... would the reply be sent as echomail with a flag or is it a reply to the local echomail JAM base that can only be read by the user it's addressed logged into the BBS reading that base?

    --- Mystic BBS v1.12 A44 2020/01/29 (Windows/32)
    * Origin: Agency BBS | Dunedin, New Zealand | agency.bbs.nz (21:1/101)
  • From g00r00@21:1/108 to Avon on Fri Jan 31 13:00:22 2020
    Oh OK, golly.. my memory fails me on that one. I'm not sure how the From

    I think it might have just been one of those things you were passing along to me that someone had brought up (Netmails addressed to invalid users). Or my memory is failing me! I've been starting to notice my age lately *cry*

    and Sent would help? I'm picking if I sent a netmail so I am a known
    user then the message would not be purged even if the To field was to someone outside the BBS - am I reading you right?

    Yes, thats the idea. But not taking into consideration the From field was an oversight on my part and I have already corrected that in the latest pre-alpha. People may still want to see their old Netmail messages that they've sent after all.

    It does seem a bit strange to me still that this purge process would be applied to Netmail style bases given the nature of Netmail being often
    to be one of sending to others who may not be a member of your own BBS.

    The reason Netmail is included is because its private. A Netmail message that comes *from* your BBS should always be from a BBS user. A netmail message *to* your BBS should always be addressed to a BBS user...

    When neither of those things are true, you are left with a private message that no one can ever see. Those are the messages the purge is trying to remove. If you have a use-case that you think falls outside of that idea please let me know. I certainly don't have the same experience with FTN as you.

    I use it daily to maintain Usenet bases at a max number of messages... so it's not really an option for me to cease this.

    I understand. It should be corrected now to include the From field. If it still seems like something you absolutely don't want, then I can make it an option in the mutil.ini for you to disable.

    Private reply how? Sorry having a blond moment... would the reply be
    sent as echomail with a flag or is it a reply to the local echomail JAM base that can only be read by the user it's addressed logged into the
    BBS reading that base?

    Private reply is meant to be a local thing so you wouldn't enable it for echomail bases. Some of those QuickBBS style softwares didn't have the concept of a private mailbox, they just had message bases and thats it. In a base you could reply with a private flag so only the "To" user could read the message, and that would be how you would communicate privately. Private Reply is intended to mimic that behavior.

    If you did try to enable it for echomail bases, I don't know how the various systems would handle it. But I think Mystic would ask the user if they wanted to send the message as Private and it would send it out with that flag (untested, just guessing)

    --- Mystic BBS v1.12 A44 2020/01/29 (Linux/64)
    * Origin: Sector 7 (21:1/108)
  • From Captain Obvious@21:1/157 to g00r00 on Fri Jan 31 16:32:20 2020
    On 31 Jan 2020, g00r00 said the following...

    latest pre-alpha. People may still want to see their old Netmail
    messages that they've sent after all.

    I do for one. I have a tendency to go away after a few months and leave
    things on auto while I work on other projects. Makes it easier for me to look up areafix passwords and the like when I roll back around to bbsing.

    -=>Richard Miles<=-
    -=>Captain Obvious<=-
    -=>bbs.shadowscope.com<=-

    --- Mystic BBS v1.12 A44 2020/01/31 (Windows/32)
    * Origin: Shadowscope BBS | bbs.shadowscope.com | Temple, GA (21:1/157)
  • From g00r00@21:1/108 to Captain Obvious on Fri Jan 31 20:21:50 2020
    I do for one. I have a tendency to go away after a few months and leave things on auto while I work on other projects. Makes it easier for me to look up areafix passwords and the like when I roll back around to bbsing.

    A little hint for Areafix and stuff. You can send Areafix, Filefix and
    Netmail from the Echomail editor using the / menu so you don't have to know
    the passwords! I think there was a bug with it depending on how you had your Netmail area configured in A43 but its since been fixed.

    But I agree with you I just spaced on the idea that I should have it check
    the From field when I added that. It should be fixed up now!

    --- Mystic BBS v1.12 A44 2020/01/29 (Linux/64)
    * Origin: Sector 7 (21:1/108)
  • From Avon@21:1/101 to g00r00 on Sat Feb 1 21:27:08 2020
    On 31 Jan 2020 at 01:00p, g00r00 pondered and said...

    users). Or my memory is failing me! I've been starting to notice my
    age lately *cry*

    I have updated to the latest build and just catching up on emails.
    Seems some of the message I wrote here at 1/101 did not come back to me on
    the rescan, not sure why, but they were only a few recent ones I wrote to you and others... I can see the replies to my posts but not my originals. I guess because they came from here MUTIL would not toss them back in despite the echomail base not containing them?

    + Feb 01 20:29:07 Importing 0108f60f.pkt (21:1/100 to 21:1/101)
    + Feb 01 20:29:07 Import #75041 to FSX_GEN
    + Feb 01 20:29:07 Import #75042 to FSX_GEN
    ! Feb 01 20:29:07 Circular PATH reference
    ! Feb 01 20:29:07 Duplicate message found in FSX_GEN

    Yes, thats the idea. But not taking into consideration the From field
    was an oversight on my part and I have already corrected that in the latest pre-alpha. People may still want to see their old Netmail
    messages that they've sent after all.

    Thanks for this.

    The reason Netmail is included is because its private. A Netmail
    message that comes *from* your BBS should always be from a BBS user. A netmail message *to* your BBS should always be addressed to a BBS user...

    When neither of those things are true, you are left with a private
    message that no one can ever see. Those are the messages the purge is trying to remove. If you have a use-case that you think falls outside of that idea please let me know. I certainly don't have the same
    experience with FTN as you.

    All sounds good. One thought, consider if a script can send netmail does it need to be as a user also or is it conceivable it may do this outside the
    user base records? If so then that could be some exception I can conceive but
    I agree it's a rare one.

    I use it daily to maintain Usenet bases at a max number of messages.. it's not really an option for me to cease this.

    I understand. It should be corrected now to include the From field. If it still seems like something you absolutely don't want, then I can make it an option in the mutil.ini for you to disable.

    I think it's fine now as it is. If I find anything down the track to suggest otherwise I'll let you know. Cheers.

    Private reply is meant to be a local thing so you wouldn't enable it for echomail bases. Some of those QuickBBS style softwares didn't have the concept of a private mailbox, they just had message bases and thats it. In a base you could reply with a private flag so only the "To" user
    could read the message, and that would be how you would communicate privately. Private Reply is intended to mimic that behavior.

    I would not offer it for echomail bases as I think it's confusing. I also wonder about the worth of it to be honest.

    If you did try to enable it for echomail bases, I don't know how the various systems would handle it. But I think Mystic would ask the user
    if they wanted to send the message as Private and it would send it out with that flag (untested, just guessing)

    Likewise, untested but assuming yes, and another reason to remove from
    echomail bases or all together?

    Best, Paul

    --- Mystic BBS v1.12 A44 2020/01/31 (Windows/32)
    * Origin: Agency BBS | Dunedin, New Zealand | agency.bbs.nz (21:1/101)
  • From Vk3jed@21:1/109 to g00r00 on Sat Feb 1 20:58:00 2020
    On 01-31-20 13:00, g00r00 wrote to Avon <=-

    If you did try to enable it for echomail bases, I don't know how the various systems would handle it. But I think Mystic would ask the user
    if they wanted to send the message as Private and it would send it out with that flag (untested, just guessing)

    It should get sent as private. Private echomail is a "thing". I used to have to use it on some networks that were gated to QWK, back in the early 90s. Private echomail was the only way priate messages could be exchanged between users on different BBSs. :)

    These days, less of a need, with Synchronet's QWK extensions allowing QWK netmail.


    ... Question of the day: x² + ¼x² - x = 16 [solve for x]
    === MultiMail/Win v0.51
    --- SBBSecho 3.10-Linux
    * Origin: Freeway BBS Bendigo,Australia freeway.apana.org.au (21:1/109)
  • From Captain Obvious@21:1/157 to Avon on Sat Feb 1 11:22:05 2020
    On 01 Feb 2020, Avon said the following...

    could read the message, and that would be how you would communicate privately. Private Reply is intended to mimic that behavior.

    I would not offer it for echomail bases as I think it's confusing. I also wonder about the worth of it to be honest.

    I can see the use of it in local bases. Not a very common occurrence but if I want to Reply to a message privately but also be able to quote for context
    then it would work. Otherwise I would have to export the message and
    re-upload into email in order to be able to quote. Like I said, don't think it's something that's used often but...

    -=>Richard Miles<=-
    -=>Captain Obvious<=-
    -=>bbs.shadowscope.com<=-

    --- Mystic BBS v1.12 A44 2020/01/31 (Windows/32)
    * Origin: Shadowscope BBS | bbs.shadowscope.com | Temple, GA (21:1/157)