On 08 Feb 2020 at 12:43p, g00r00 pondered and said...

For the echomail side, I think it'd be nice to hear what reports would
be nice to have regardless of if the data is there or not. At the very least it may help me sort out how to design the template engine for
report generation.

Ideally all reports to be defined in terms of a start date/time and end date/time range. Options allowing to quickly generate reports based on a last x minutes, or x hours or x days. An option to show all data stored since last reset. Options to enable available report fields to be added or removed. Options to be able to vary sort order of the fields used in the templates.

Brain dump of some stuff you might want to capture and allow for the following to be displayed in reports, over and above what you have now...

Mystic Version and OS of each Node (it's being reported now in BinkP and Fidopoll) and/or any mailer name supplied by a node not using Mystic.

A record of the last 5 - 10 echomail files sent to a node, file name, size, date sent, success or failure of transmission. The same thing for files received from a node.

A record of the last 5 - 10 files in filebox sent to a node with same stats as per echomail traffic above. The same thing for any files/tics received by a node.

A record of the last date/time 'Unable to connect' was reported for a node,
and a stats count of this info per node

A record of the last date/time 'Connection lost' or 'Authorization failed'
was reported for a node, and a stats count of this info per node.

Records of Areafix / Filefix requests sent in by a node. To be able to pull/post a report that shows nodes xyz sent x Areafix requests today, date/time of them, if request was successful or not, perhaps a summary of actions performed e.g rescan d=999 .. that kind of thing.

As for the reports

1) Node Activity - each node showing the data you now collect, one node per line, with options to change the order of the data, add/remove fields, choose the sort order of the output by a selected data field used.

Nice to have options (perhaps for this report and others...) would be to also show a nodes Export Type and Archive Type and Session Type along with address being polled (if not set to HOLD), Also include options to show (or hide) echomail nodes by echomail group or access flags assigned to the node.

2) Node Inactivity - a report the shows inactivity by oldest to newest. Not sure how inactive should be defined. Guess there should be options for
inactive inbound and inactive outbound traffic. Show this in terms of hours
or days last heard from based on stats collected. Express this info
optionally in terms of hours, days, weeks etc. in the reports

3) Problem reports - show reports of the nodes with 'Unable to Connect' etc. style issues. Perhaps one report for each type of error. Data to show includes node number, first date issue encountered (express in days since error
began?)

4) Informational reports - node info showing when last heard from, last X
files sent and received by the node. A ranking option to show which nodes are most active in terms of traffic in/out etc.

This makes me think about the JAM base tools I have been using (with mixed success lately) that will read a base and dump more echomail specific reports out that MUTIL can post. Would you be open to creating some of those reports
as part of this tranche of work? I can send you example outputs of what I
have been able to create in the past. The issue seems to be (I think) the JAM bases are getting so full that some stats tools are baulking at reading them now :(

I'll think of more no doubt but it's late as I type this and i need to be up
at 6am. so will stop there for now.

--- Mystic BBS v1.12 A44 2020/02/04 (Windows/32)
* Origin: Agency BBS | Dunedin, New Zealand | agency.bbs.nz (21:1/101)

Mystic Version and OS of each Node (it's being reported now in BinkP and Fidopoll) and/or any mailer name supplied by a node not using Mystic.

I hadn't thought of tracking the mailer. I've made a note of this.

A record of the last 5 - 10 echomail files sent to a node, file name, size, date sent, success or failure of transmission. The same thing for files received from a node.

I already have a system like this in place although it does not track a transfer failure, it only logs the time and name of files sent or received successfully. I'll see if I can track the fail cases too.

I think I accidentally left it enabled in one of the A44 pre-alphas (see if
you have any echodata*.dat files in the data directory for example). This individual node logging is/was/will be viewable in the Echnode editor and
will reset along with the other statistics.

A record of the last date/time 'Connection lost' or 'Authorization
failed' was reported for a node, and a stats count of this info per node.

For all of the various error stuff, the way I was planning to track it was to have an overall "Error" count and then a "last error type" which just gives you whatever the last error was (cannot connect, authentication failed, session timeout, session lost). I felt that this was a much more manageable way to track and it and still gives us a view into nodes that may have connection issues and what those issues are. Let me know what you think on that.

Records of Areafix / Filefix requests sent in by a node. To be able to pull/post a report that shows nodes xyz sent x Areafix requests today, date/time of them, if request was successful or not, perhaps a summary of actions performed e.g rescan d=999 .. that kind of thing.

I'll make a note of this although I am not sure what exactly I will do here.

This makes me think about the JAM base tools I have been using (with
mixed success lately) that will read a base and dump more echomail specific reports out that MUTIL can post. Would you be open to creating some of those reports as part of this tranche of work? I can send you example outputs of what I have been able to create in the past. The

Yes please do! Show me the tool you use and the output if you can. I think I used to use one too that did something similar (I probably got it from you) but it was so long ago that I don't remember.

JAM is nice because of its legacy support, but it certainly has its issues and will have date problems and issues with scaling to large amounts of messages in the future.

I've considered creating a "JAM64" extension that would address some of them, but that would break legacy support. Or just using my own format.

--- Mystic BBS v1.12 A45 2020/02/09 (Linux/64)
* Origin: Sector 7 (21:1/108)

On 09 Feb 2020, g00r00 said the following...

Mystic Version and OS of each Node (it's being reported now in BinkP Fidopoll) and/or any mailer name supplied by a node not using Mystic.

I hadn't thought of tracking the mailer. I've made a note of this.

I just had another thought on the echomail tracking. If there was a way to
see what each node had waiting for them to pick up, it would help the hub operations.

For example, if a node stops polling, it would show 593 waiting files.

While I was using Husky, I had it set up to put outbound mail into fileboxes. By doing that, I could look in each directory and see what was waiting for
each node.

Just a thought. :)


---

Black Panther(RCS)
Castle Rock BBS

--- Mystic BBS v1.12 A43 2019/03/02 (Linux/64)
* Origin: Castle Rock BBS - bbs.castlerockbbs.com (21:1/186)

On 09 Feb 2020 at 11:53a, Black Panther pondered and said...

I just had another thought on the echomail tracking. If there was a way
to see what each node had waiting for them to pick up, it would help the hub operations.

For example, if a node stops polling, it would show 593 waiting files.

I would love a report that looks at the echomail\out folders and summarises
the contents of what is in there by node. And something that also identifies orphaned files in there. My hunch at times is that there are some in there
but I have no way of know what is associated with what etc..

--- Mystic BBS v1.12 A44 2020/02/04 (Windows/32)
* Origin: Agency BBS | Dunedin, New Zealand | agency.bbs.nz (21:1/101)

On 09 Feb 2020 at 12:50p, g00r00 pondered and said...

I hadn't thought of tracking the mailer. I've made a note of this.

Cool, thanks :)

A record of the last 5 - 10 echomail files sent to a node, file name, size, date sent, success or failure of transmission. The same thing f files received from a node.

I already have a system like this in place although it does not track a transfer failure, it only logs the time and name of files sent or
received successfully. I'll see if I can track the fail cases too.

I think tracking both success and fail would be good, thanks for looking at this.

A record of the last date/time 'Connection lost' or 'Authorization failed' was reported for a node, and a stats count of this info per n

For all of the various error stuff, the way I was planning to track it
was to have an overall "Error" count and then a "last error type" which just gives you whatever the last error was (cannot connect,
authentication failed, session timeout, session lost). I felt that this was a much more manageable way to track and it and still gives us a view into nodes that may have connection issues and what those issues are.
Let me know what you think on that.

I agree an overall count will likely best show someone at a gross level if there are issues or not. I'm still left wondering if errors are created by a mix of issues how best to quickly identify those? Being able to drill down somehow by error type would be advantageous for fault diagnosis else risk missing something. Guess we can have a play with this and build things out as needed.

Records of Areafix / Filefix requests sent in by a node. To be able t pull/post a report that shows nodes xyz sent x Areafix requests today date/time of them, if request was successful or not, perhaps a summar actions performed e.g rescan d=999 .. that kind of thing.

I'll make a note of this although I am not sure what exactly I will do here.

I think for nodes to be able to request this info via Areafix and Filefix command it would be as helpful to them to have access to this data as it
would be for a HUB to be able to view it and/or publish aggregate totals of some of this.. Yeah have a ponder, it's just a idea :)

This makes me think about the JAM base tools I have been using (with mixed success lately) that will read a base and dump more echomail specific reports out that MUTIL can post. Would you be open to creati some of those reports as part of this tranche of work? I can send you example outputs of what I have been able to create in the past. The

Yes please do! Show me the tool you use and the output if you can. I think I used to use one too that did something similar (I probably got
it from you) but it was so long ago that I don't remember.
JAM is nice because of its legacy support, but it certainly has its
issues and will have date problems and issues with scaling to large amounts of messages in the future.

I will find some old reports and post them here and send you the dated
software to have a look at. From memory one has the source code so you may be able to rework something that suits Mystic needs.

I've considered creating a "JAM64" extension that would address some of them, but that would break legacy support. Or just using my own format.

Given it's 2020 I personally have no issue with using something in this
decade vs decades ago :)

--- Mystic BBS v1.12 A44 2020/02/04 (Windows/32)
* Origin: Agency BBS | Dunedin, New Zealand | agency.bbs.nz (21:1/101)

I just had another thought on the echomail tracking. If there was a way
to see what each node had waiting for them to pick up, it would help the hub operations.

I want to add in some sort of "packet browser" I think into the echonode
editor or maybe somewhere else, but I haven't really sorted out how I want it to work in my brain yet lol.

While I was using Husky, I had it set up to put outbound mail into fileboxes. By doing that, I could look in each directory and see what
was waiting for each node.

Although I haven't actually tried this specific case, you could probably do the same thing with Mystic's directory tossing. Although I wouldn't really recommend it as its not really a standard practice.

--- Mystic BBS v1.12 A45 2020/02/09 (Linux/64)
* Origin: Sector 7 (21:1/108)