When I get files from Frank for his SciNet network the BinkP server has conniptions and borks
If you look at my echomail\in it's full of files when the session failed and new files were created.
4/01/2020 01:52 a.m. 49,397 sciibbs.1.zip
4/01/2020 01:52 a.m. 1,431,901 sciibbs.10.zip
Seems for whatever reason this large file is not liked..
I'm still on A43 here running under Linux/64 and I get these files fine. Never had an issue with them so far, if that can help in tshooting.
4/01/2020 01:52 a.m. 49,397 sciibbs.1.zip
4/01/2020 01:52 a.m. 1,431,901 sciibbs.10.zip
Seems for whatever reason this large file is not liked..
I'm still on A43 here running under Linux/64 and I get these files fine. Never had an issue with them so far, if that can help in tshooting.
Yep it's weird.. I'm not sure but perhaps it's OS specific, or something on my system. It's only those larger files it seems... well that's the only thing that large I get at the moment.. so yeah, figured perhaps it was related to size.. Hmm..
Yep it's weird.. I'm not sure but perhaps it's OS specific, or something on my system. It's only those larger files it seems... well that's the only thing that large I get at the moment.. so yeah, figured perhaps it was related to size.. Hmm..
I'm still on A43 here running under Linux/64 and I get these files fi Never had an issue with them so far, if that can help in tshooting.
Yep it's weird.. I'm not sure but perhaps it's OS specific, or something on my system. It's only those larger files it seems... well that's the only thing that large I get at the moment.. so yeah, figured perhaps it was related to size.. Hmm..
I have checked with most of my downlinks and most of the nodes in Fidonet that get the Filegate fileecho that I distribute them on and no one else has had these issues. I wonder if it's a packet routing issue.
I was reading g00r00's answer regarding error correction vs binkd and I was listening to Total FM this PM and noticed regular chops in the audio and it continuously had to buffer. Maybe this is not related but it may help to confirm the packet drop part. Maybe some network congestion or other errors. Just a thought...
I was reading g00r00's answer regarding error correction vs binkd and was listening to Total FM this PM and noticed regular chops in the au and it continuously had to buffer. Maybe this is not related but it m help to confirm the packet drop part. Maybe some network congestion o other errors. Just a thought...
Well firstly thanks for listening.. it's appreciated :) Next, Hmm... I'm running a fibre 1000 down 500 up plan so hearing there's choppy audio is
a worry. I wonder if it was the gear processing it? The again I am way
way away here in little New Zealand which does happen to be about as far away as anyone who wants to get away from stuff can go :)
I was reading g00r00's answer regarding error correction vs
binkd and I was listening to Total FM this PM and noticed
regular chops in the audio and it continuously had to buffer.
Maybe this is not related but it may help to confirm the packet
drop part. Maybe some network congestion or other errors. Just a thought...
binkp uses TCP which "provides reliable, ordered, and error-checked delivery of a stream of octets (bytes) between applications" (quote from wikipedia). It is as reliable as email delivery or an http download. You normally don't have garbled text in emails or web pages, because of transmission errors.
:)binkp uses TCP which "provides reliable, ordered, and
error-checked delivery of a stream of octets (bytes) between
applications" (quote from wikipedia). It is as reliable as
email delivery or an http download. You normally don't have
garbled text in emails or web pages, because of transmission
errors.
Hehe don't worry, I know TCP quite well ;)
However, it may have been caused by dropped connections, which
TCP does not protect against.
All in all, not looking for a network course here but just
trying to provide input to help in the resolution... I'm pretty
sure my network is good, don't worry ;)
However, it may have been caused by dropped connections, which
TCP does not protect against.
A dropped connection shouldn't be a problem, because binkp sends the
size of the file in advance. If the connection drops and not all bytes have been received, the client should recognize the file transfer as failed. If there are incomplete files caused by a dropped connection, we should consider the possibility of a bug in the server or client implementation. If it is caused by something else, it might be a bug too.
However, it may have been caused by dropped connections,which
TCP does not protect against.
A dropped connection shouldn't be a problem, because binkp
sends the size of the file in advance. If the connection drops
and not all bytes have been received, the client should
recognize the file transfer as failed. If there are incomplete
files caused by a dropped connection, we should consider the
possibility of a bug in the server or client implementation. If
it is caused by something else, it might be a bug too.
Well, I agree there must be a problem or a bug. Where I disagree
is the fact you exclude some possibilities out of the equation
right from the start based on "is not supposed to" or "should do
xyz" in your explanation. This is not a good troubleshooting
approach to the problem IMO.
Every possibilities should be considered and analyzed, starting
with the most probable and go down to the next and so on...
It was mainly a response to the bullshit explanation that garbage in the log file and corrupted files are to be expected on droppped connections, because binkp has no error correction.
"I think (from memory) just swapping out to BinkD sorted the issues Oli was reporting.. now I am back to using MUTIL for tossing but still BinkD at this stage."
--- The 💩 Tosser v1.12 A44
* Origin: 🦄🌈 (21:1/151)
Well, I agree there must be a problem or a bug. Where I disagree is the fact you exclude some possibilities out of the equation right from the start based on "is not supposed to" or "should do xyz" in your explanation. This is not a good troubleshooting approach to the problem IMO.
The fact is that files are actually saved incomplete, therefore there is an issue. The process that receives the file from the network cannot do it's job completely, what remains to be answered is what between that piece of software and the network (or both maybe) is acting up.
let's start with the most probable one: there is a bug in mystic bbs' binkp implementation.
Well, I'm not sure how garbage in the log files would appear if a
problem is on the remote side. The cause for a logging garbage issue has to be on the "logging" side software, maybe a lack of output
sanitization or some more serious problems like memory overwrite, buffer overflow, etc.
It was mainly a response to the bullshit explanation that
garbage in the log file and corrupted files are to be expected
on droppped connections, because binkp has no error correction.
Well, I'm not sure how garbage in the log files would appear if
a problem is on the remote side. The cause for a logging garbage
issue has to be on the "logging" side software, maybe a lack of
output sanitization or some more serious problems like memory
overwrite, buffer overflow, etc.
According to the snippet here...
"I think (from memory) just swapping out to BinkD sorted the
issues Oli was reporting.. now I am back to using MUTIL for
tossing but still BinkD at this stage."
... Avon is still using BinkD for the packet transfers. Netsurge
is also running BinkD as far as I can tell. So if both are
running BinkD, how can Mystic be the most probable cause in the
failed transfers issues Avon is having, log garbage and what
not? It looks to me that Mystic has nothing to do with this
since it's not even in the path anywhere between the two.
Yep, I know I am preaching to the choir here but TCP is only a small
part of the equation. To try to make a statement that claims that it makes something bullet-proof is fundamentally short-sighted.
If you're talking about Mystic the logging garbage is intentional.
Mystic is dumping the entire data buffer because the frames were out of sync. The intention is that we can look at the logs and see what
content was being sent to try to determine the issue.
The log is clearly not from BinkD, which uses a different format. If you read the whole log you see that it is a session between BinkD and Mystic
If there is a problem with transmission that amounts to a bug then it should be consistent and reproducible. As far as I know that isn't the case here and with the frequency that Mystic sends file it would be a wide-spread issue.
I have most likely assumed Avon was running BinkD on both the hub and Agency, he might be running BinkD only on the Hub and Agency still runs Mystic BinkP.
If that is intentional for debugging purposes, this is fine by me. That may be a good idea to dump these debug packet in a separate file since they can break log processing tools (like one I'm using for my setup). That way you have the debug data AND the main log file remains clean.
Just my 2c.
The debug logging for BINKP and socket error message is only temporary.
I don't think it changes the format of the log, it just adds additional lines. The only time it would balloon like that is when it receives jibberish from the remote BinkP client so it shouldn't happen regularly. Really I am too lazy to separate it right now ;)
Ignore that troll trying to make issues. This is the same guy who is changing message "To" fields from g00r00 to p00p and message tear lines
to "Garbage v1.12" lol Dude is here for the wrong reasons and has legit issues, sadly.
If Avon can reproduce an actual transfer issue regularly then we can diagnose it, but until that happens there isn't much I can do do. I
don't seem to have any problems on my own BBS with my binkd uplinks so that path has been a bust.
Seeing one of his replies, apparently it happens at regular interval
from the same source, so maybe there will be a way to reproduce to pinpoint where the issue is.
That would be great if so! I did set up binkd today and spent some time sending a 700 megabyte ISO back and forth between Mystic MIS and a BBS with binkd.
You're correct Agency is running Mystic BinkP and the 1/100 HUB
is using BinkD at the moment. Because the MIS logs roll over so
quick I may not have anything I can share until the next time it
happens.. but it does seem to happen each time these files are
sent.
I have found the mis.1.txt file contains one example of the error and
also another in my fidopoll to Frank. So it happens both ways between these two systems.
But really, unless someone has something reproducible its hard to enact
on it as anything outside of a connection hiccup. I get these reports from time to time and we've never pinpointed any issue.
The logs I saw (correct me if I'm wrong) were all A43 right?
I noticed the binkp debug logging isn't on and also A43 is very old is there any way we can test with the latest?
I would suggest you remove the node logs and if they're building on a current Jan 29 version then lets look into it.
I agree. This problem is localized to Avon and myself only. I have
plenty of other Mystic downlinks that get the same file just fine. That leads me to believe it is a TCP issue, albeit routing or packet loss,
hard to pinpoint unless Avon and I do some extensive testing which is
what I might just do.
I agree. This problem is localized to Avon and myself only. I have
plenty of other Mystic downlinks that get the same file just fine. That leads me to believe it is a TCP issue, albeit routing or packet loss,
hard to pinpoint unless Avon and I do some extensive testing which is
what I might just do.
On 01-29-20 13:49, g00r00 wrote to Netsurge <=-
I can't seem to reproduce anything locally between Mystic/BinkD and
also my Fido/Agoranet uplink is BinkD and I think its been years since we've had any issues.
I appreciate the help! I was actually wondering if there could be something with the actual file but I guess not since you mentioned its fine with other connections.
Thanks sir :) I'm going to try to update Agency tonight to the latest A44 build at which time it will be interesting to see how get on when the
next round of files are sent out. Can you let me know how many days
until that happens? I can then keep an eye out for them.
A bug has just been found in binkd that may or may not be relevant.
From what I recall, memory is being freed before being read, and if malloc() is being aggressive about reusing newely freed memory, then things get corrupted. This may also be the issue I struck with Avon's
Fingers crossed. :) Sorry if the newer stuff causes any additional
issues but lets hope there is some progress somewhere.
A bug has just been found in binkd that may or may not be
relevant.
On 01-29-20 21:13, g00r00 wrote to Vk3jed <=-
A bug has just been found in binkd that may or may not be relevant.
From what I recall, memory is being freed before being read, and if malloc() is being aggressive about reusing newely freed memory, then things get corrupted. This may also be the issue I struck with Avon's
This is good to know. It certainly could be a non-Mystic issue. There were a handful of those over the years from MBSE, IREX, and all of the Taurus line of mailers.
On 01-29-20 22:12, ryan wrote to Vk3jed <=-
[snip]
I'm not much of a Pascal guy but does Pascal have malloc()?
On 01-30-20 09:10, Oli wrote to Vk3jed <=-
30 Jan 20 12:04, you wrote to g00r00:
A bug has just been found in binkd that may or may not be
relevant.
It's only about node entries with a non-standard port number, I don't think it's related. But we still should ask the question, is it only happening with systems that listen on a non-standard port number? Or
test it: just apply the patch and see if it's still happening.
Here is the link to the pull request:
https://github.com/pgul/binkd/pull/16
A bug has just been found in binkd that may or may not be
relevant.
It's only about node entries with a non-standard port number, I
don't think it's related. But we still should ask the question,
is it only happening with systems that listen on a non-standard
port number? Or test it: just apply the patch and see if it's
still happening.
That would explain the issue I had with 1/100. :)
Here is the link to the pull request:
https://github.com/pgul/binkd/pull/16
The issue I found was that the upgrade would not move the text, menu and scripts dir over to the default theme. I had to manually doing it. The
BBS seems to be working OK and I am hopeful the rest worked alright.
Time will tell and I have a backup.
The issue I found was that the upgrade would not move the text, menu and scripts dir over to the default theme. I had to manually doing it. The
BBS seems to be working OK and I am hopeful the rest worked alright.
Time will tell and I have a backup.
Ahh crap I was working on changing it from a move to a copy and I think I got interrupted. I'll take a look at that right away!
I agree. This problem is localized to Avon and myself only. I have plenty of other Mystic downlinks that get the same file just fine. Th leads me to believe it is a TCP issue, albeit routing or packet loss, hard to pinpoint unless Avon and I do some extensive testing which is what I might just do.
Thanks I would appreciate another set of eyes on it. I have enabled the intense binkP logging in the A44 pre-releases that may or may not help (assuming Avon has the means to capture it)
I can't seem to reproduce anything locally between Mystic/BinkD and also my Fido/Agoranet uplink is BinkD and I think its been years since we've had any issues.
On 01-30-20 14:49, Oli wrote to Vk3jed <=-
Just to avoid stupid rumors about binkd:
The bug caused an error on non-standard port number and the connection failed. The issue that was reported didn't mention successful
connections and failed / incomplete file transfers.
I appreciate the help! I was actually wondering if there could be something with the actual file but I guess not since you mentioned its fine with other connections.
Thanks sir :) I'm going to try to update Agency tonight to the latest A44 build at which time it will be interesting to see how get on when the
next round of files are sent out. Can you let me know how many days
until that happens? I can then keep an eye out for them.
I'd like to put some time aside and do some testing between you and me, directly. Some trace routing and a few other packet tests, just to see
if we can nail down this issue.
I appreciate the help! I was actually wondering if there could be something with the actual file but I guess not since you mentioned it fine with other connections.
I think if the binkp protocol has some form of error correction this wouldn't be an issue. I have 30+ downlinks and Avon is the only one with the issue.
I'd like to put some time aside and do some testing between you and me, directly. Some trace routing and a few other packet tests, just to see
if we can nail down this issue.
| Sysop: | Eric Oulashin |
|---|---|
| Location: | Beaverton, Oregon, USA |
| Users: | 128 |
| Nodes: | 16 (0 / 16) |
| Uptime: | 75:10:25 |
| Calls: | 1,528 |
| Calls today: | 5 |
| Files: | 2,153 |
| Messages: | 313,811 |
| Posted today: | 2 |