That said, this is a Done Thing.
So, when I needed to transfer some large (well, larger than the 5 MB permissible per email message) to Veritas from $CURRENT_EMPLOYER, and I knew that they had an explicitly listed FTP proxy in the web browser auto-configuration file, I was a bit surprised to find that it simply did not work. Didn't work when I told WS_FTP.EXE to use the proxy in its firewall configuration tab, didn't work when I played dumb and attempted to FTP through Internet Explorer, and wasn't failing just because I was connecting to a write-only directory over at Veritas; it just doesn't work. (Please don't suggest what you think the problem is here. My bet's on broken NAT in a Checkpoint firewall, but I don't really care. The solution, for me, is "open a ticket".)
Fine, whatever. I'll just FTP it up from home.
And, lo and behold... that doesn't work either! And now it is my fault!
Ah, the wonderful world of ipfilter's obtuse method of building rules, wherein the software does what you say as obstinately as possible, rather than making some obvious assumptions about what you meant. It took some glancing back and forth between this and my very own /etc/ipnat.conf before the head-slapper moment. What they've got:
For NAT and ftp clients behind NAT to work, add the following minimum mappings to /etc/ipnat.conf:What I had (for the 802.11b subnet; 10.0.0.0/24 is wired):
# Use ipfilter ftp proxy for ftp client transfers mode: active
map ep1 192.168.1.0/24 -> 0.0.0.0/32 proxy port ftp ftp/tcp
# Map all tcp and udp connections from 192.168.1.0/24 to external IP address,
# changing the source port number to something between 40,000 and 60,000 inclusive
map ep1 192.168.1.0/24 -> 0.0.0.0/32 portmap tcp/udp 40000:60000
# For all other IP packets, map to the external IP address
map ep1 192.168.1.0/24 -> 0.0.0.0/32
Make sure all the `proxy' lines are before any generic `portmap' lines, as the first match always wins.
map sf0 10.0.1.0/24 -> 66.92.234.98/32 portmap tcp/udp 40000:42999What's wrong with that picture?
map sf0 10.0.1.0/24 -> 66.92.234.98/32
map sf0 10.0.1.0/24 -> 66.92.234.98/32 proxy port ftp ftp/tcp
Well, here's what works:
map sf0 10.0.1.0/24 -> 66.92.234.98/32 proxy port ftp ftp/tcpThe difference there is purely one of order, and given that I knew that both ipnat and ipf applied rules in a "first match" approach, this can be said to be my mistake.
map sf0 10.0.1.0/24 -> 66.92.234.98/32 portmap tcp/udp 40000:42999
map sf0 10.0.1.0/24 -> 66.92.234.98/32
But that's dumb!
Sure, I can conceive of some cases in which there is no clear choice, and in that case falling back to first-match might be fine, but in this case the rule that was simply never being matched (the ftp proxy) is a clear subset of the larger rule that was being matched first (the overall map for the IP address range), and it's counter-intuitive to have more specific rules for the same set of number space come before more general ones. Or, if you want to get really clever, how 'bout we go do another Done Thing and do the same damn thing that every OS out there does for routing: longest match! You know, considering we're effectively doing routing here anyway, and that's the expected behavior for configuring routes...
And it's not like this is hard to change, either! ipnat's not rereading this configuration file for every packet that comes in (that would be abysmally slow, would mean that you could make config changes on the fly without stopping/restarting ipnat, and is really not what the code's doing--I've looked)... it's parsing this config file once, at startup, building a state machine off that, and then matching packets against its in-memory, indexed-for-speed state machine. That means you're sitting there, at the time that you're building the state machine, and you really can notice that this rule will simply never be applied, and it will never be applied because it comes after this other rule for the same goddamn address range.
As with any time that you're doing something slightly other than what the user said, you should clearly print a warning message in this case. And, clearly, there's a limit to the breakage you can save the user from here, but it's really not that hard to make a very small change to the parsing mechanism that blows away 90% of the simple errors and gets the user what they actually wanted.
Which brings up another point: you can traverse this state machine and say, "Hey, this rule will never be matched." In fact, if you're optimizing for efficiency, you're doing that anyway and throwing away unmatched rules so that the memory footprint of your state machine is smaller. Not warning the user of unmatched rules is lazy enough to be called grossly negligent. Even that would be enough to have smacked me upside the head the first time I misconfigured this.
Bullshit like this is what infuriates me about Windows (where, of course, it happens way the hell more often). Tell me what you're fucking doing, you damn computer. I'm the user: you work for me, not the other way 'round.
June 28 2005, 14:32:56 UTC 6 years ago
Cosmic.
I have no idea what you just said, but as I was reading it, someone in the next room in my office was explaining to someone that their firewall might be the reason that they were having trouble with our ftp.June 28 2005, 23:32:22 UTC 6 years ago
Re: Cosmic.
I find this confirmation that there are a bunch of fuckheads in the world, and, given the number of times I've encountered exactly this problem at huge corporations who should know better, not all that surprising.June 28 2005, 15:05:31 UTC 6 years ago
June 28 2005, 20:17:04 UTC 6 years ago
But i think the worse, most horrible thing ever is me, now senior tech at $ISP_CALLCENTER having to explain to the other techs wtf is the difference between regular ftp and passive FTP, and why passive is a good thing to use if you're behind a NATed connection.
I think i definitly needto find myself another job now. It's getting too ridiculous for words.
June 28 2005, 21:02:20 UTC 6 years ago
Really the time has come to admit that FTP and "the modern Internet" (viz, many stateful firewalls, and NAT devices) simply do not play nicely together, and use something else instead. HTTP and WEBDAV would probably be my choice.
FTP. Just say no.
Alas, all the same problems arise all over again with pretty much every VoIP protocol, which also use a signalling channel, plus arbitrarily chosen media channels. And the media channels, just like FTP, can easily end up going places that the signalling channel didn't. This is true of H.323 (which is understandable, it was designed by telcos) and also true of SIP (which is less forgiveable as it was designed by "Internet" folks).
Clearly we haven't learnt anything in the interveening 25 years.
Ewen
June 28 2005, 23:28:56 UTC 6 years ago
What we need is all these fuckheads to, you know, maybe follow RFCs 1918 and 2979 once in a blue moon. Speaking of which, my license plate is up for renewal, but I don't think I can get either, because they could, theoretically, be generated by PA's current random gen. Alas.
There's nothing at all wrong with FTP, or SIP, or anything else you mention. They function well within establish norms that Should Work, and with which firewalls should never, ever interfere. If they do, they're broken by design.
June 29 2005, 00:02:24 UTC 6 years ago
The former (allowing arbitrary incoming connections) isn't even possible in a world filled with NAT devices, a situation forced upon us by the reluctance of "IP owning" organisations to assign meaningful numbers of IP addresses to customers (and the pain of renumbering when changing provider). (I'm reliably informed that there really isn't the shortage of IPv4 addresses that some people make out; just an unwillingness to "do the paperwork", and an unwillingness to assign "scarce" IPs for routine things like workstations.)
The latter is improbable in the face of arbitrary new protocols with arbitrary new methods for "randomly" (from the point of view of the firewall) deciding who is going to communicate with the protected hosts. FTP-aware firewalls are fairly common (albeit sometimes requiring care in setup; ipfilter is especially bad for the amount of care required). SIP aware firewalls exist (IIRC), but are rare. H.323 aware firewalls maybe. Random new protocol, unlikely.
So I'm forced to the conclusion that protocols which require communication between arbitrary devices on arbitrary ports are broken by design, at least in the face of the modern Internet. And that aside from a massive IPv6 rollout with clueful providers (hah!) and clueful customers (hah!), and a sudden disappearance of security holes in deployed application software (hah!) this situation is unlikely to change. I really don't see (stateful) firewalls going away ever. (And stateless firewalls are even more of a pain.)
The innocent days of the Internet being unprotected hosts sitting waiting for anyone that wants to connect to them are long gone. The environment that made that possible no longer exists, and isn't going to return.
Ewen
PS: In the case of SIP, et al, I really don't have a good answer. The only alternative likely to reliably work -- media proxying at each signal proxying stage -- sucks for other reasons. The best compromise might well be a "well known" media stream port.
June 29 2005, 07:15:05 UTC 6 years ago
Never mind all the firewall business, if anybody anywhere between you and the source (or destination, if you're the "client") host has sufficient access to arp-flood a router, they can pretend to be either end (which is not a new observation, mind), and quite a few protocols will happily fall prey to this.
This stuff all needs a lightweight PKI (really, just PKE--E for Exchange--would probably be enough for 90% of the cases, where you really only need authentication, not so much enciphered data transfer) protocol in place.
June 29 2005, 07:39:01 UTC 6 years ago
Alas, it also needs to be sufficiently wide spread that it is used for all such protocols rather than reinvented for each one. And of course there'd need to be some sort of "firewall discovery" method as well, so the applications could figure out who they needed to negotiate such access rights with.
There are some NAT-traversal protocols around, but I've not looked at any of them closely enough to know if they do anything better than sending outbound packets in an attempt to try to persuade the firewall to permit incoming packets due to its state table. (Or the truely ugly solution of just encapsulating everything in UDP packets from a known host.)
But I still think it'd be better to use protocols that avoid the whole problem where at all possible. FTP in particularly is considerably more complicated than a protocol for "download file from server I connected to" or "upload file to server I connected to" needs to be. (All the rest of the imagined flexibility has never really been used, or even implemented.)
Ewen
June 28 2005, 23:19:16 UTC 6 years ago
I never do!
When I use active FTP, it's my firewall that has to work right.
When I use passive FTP it's theirs. Anyone taking bets on which I can rely upon to actually fucking work? (I mean, the current example not-with-standing, since I fixed it.)
June 28 2005, 23:30:13 UTC 6 years ago
"by putting a text parser in kernel space; with total disregard for protocol layering; grotesquely insecurely""all software sucks".*eyegouge* Can't imagine what I was thinking.
June 28 2005, 23:35:52 UTC 6 years ago
June 28 2005, 17:31:12 UTC 6 years ago
June 28 2005, 18:58:46 UTC 6 years ago
June 28 2005, 19:27:10 UTC 6 years ago
June 28 2005, 23:21:36 UTC 6 years ago
I hadn't heard that you'd moved over to the Veritas NetBackup development team!
That's a big change from the PHD->academia plan we talked about this past weekend...
June 28 2005, 19:27:55 UTC 6 years ago
June 28 2005, 23:20:17 UTC 6 years ago
June 28 2005, 23:24:17 UTC 6 years ago
June 28 2005, 23:29:52 UTC 6 years ago