deprecate FTP in favor of SFTP? #2097

New Issue

zooko · 2013-11-03T18:54:23Z

zooko commented

2013-11-03 18:54:23 +00:00

There are major limitations to the LAFS-FTPd implementation—starting with the fact that mutable files just don't work—and there is no intent to fix these limitations, because the Tahoe-LAFS developers think that the SFTP protocol is better, the LAFS-SFTP implementation already works better, and we think everyone should switch from FTP to SFTP. There are more details about this here, in addition to the obvious issue that FTP lacks confidentiality and integrity: source:trunk/docs/frontends/FTP-and-SFTP.rst .

However, I've observed that people continue to use FTP because:

They think that the only difference between the two is that SFTP is encrypted, and
They are accessing it over localhost only, anyway, or they otherwise aren't worried about attackers snooping on or altering their files in flight, and
Setting up LAFS-SFTPd requires an extra step more than setting up LAFS-FTPd — you have to create an ssh keypair.

In other words, I've observed that people are unaware of the limitations and problems in the FTP protocol and the LAFS-FTPd implementation, mentioned above and documented in source:trunk/docs/frontends/FTP-and-SFTP.rst, even though we've documented them from the beginning. This is a lesson we've learned many times: it doesn't matter what the documentation says, people will continue to use a feature as long as it appears to work.

The most recent example of this pattern is the choice of Stig Atle Steffensen to use LAFS-FTPd even though I already told him that there were relevant limitations documented in FTP-and-SFTP.rst. Apparently he didn't read it, didn't notice the limitations part, or thinks those limitations are irrelevant to his use case. (Which I guess could be true for him, if he uses only ASCII filenames, only immutable files, doesn't have servers-of-happiness failures on his grid, etc.) In this tweet he wrote, it sounded like he wasn't aware of those other issues and thought that the only difference between FTP and SFTP was encryption:

“ftp is unencrypted, sftp is encrypted, but if you run everything on 'localhost' then it does not matter if you use one over the other”

— https://twitter.com/stigatle/status/397059080499789824

This ticket proposes to deprecate and then remove the LAFS-FTPd implementation in favor of LAFS-SFTPd. The justification is that LAFS-FTPd lacks important functionality, like mutable files, error reporting, and non-ASCII filenames, not to mention confidentiality and integrity, and we have no plans to add it, because the FTP protocol can't support some of those features, and because we've already implemented all of that in LAFS-SFTPd and we think anyone who uses LAFS-FTPd could (with only a little added effort) switch to LAFS-SFTPd.

I'm marking this with the tag forward-compatibility and putting it into Milestone 1.11 because if we want to leave the deprecated LAFS-FTPd functionality in place for a full major release, then not doing the deprecation notice in 1.11 will obligate us to keep LAFS-FTPd functionality running in 1.12.

There are major limitations to the LAFS-FTPd implementation—starting with the fact that mutable files just don't work—and there is no intent to fix these limitations, because the Tahoe-LAFS developers think that the SFTP protocol is better, the LAFS-SFTP implementation already works better, and we think everyone should switch from FTP to SFTP. There are more details about this here, in addition to the obvious issue that FTP lacks confidentiality and integrity: source:trunk/docs/frontends/FTP-and-SFTP.rst . However, I've observed that people continue to use FTP because: * They think that the only difference between the two is that SFTP is encrypted, and * They are accessing it over localhost only, anyway, or they otherwise aren't worried about attackers snooping on or altering their files in flight, and * Setting up LAFS-SFTPd requires an extra step more than setting up LAFS-FTPd — you have to create an ssh keypair. In other words, I've observed that people are unaware of the limitations and problems in the FTP protocol and the LAFS-FTPd implementation, mentioned above and documented in source:trunk/docs/frontends/FTP-and-SFTP.rst, even though we've documented them from the beginning. This is a lesson we've learned many times: it doesn't matter what the documentation says, people will continue to use a feature as long as it *appears* to work. The most recent example of this pattern is the choice of Stig Atle Steffensen to use LAFS-FTPd even though I already told him that there were relevant limitations documented in FTP-and-SFTP.rst. Apparently he didn't read it, didn't notice the limitations part, or thinks those limitations are irrelevant to his use case. (Which I guess could be true for him, if he uses only ASCII filenames, only immutable files, doesn't have servers-of-happiness failures on his grid, etc.) In this tweet he wrote, it sounded like he wasn't aware of those other issues and thought that the only difference between FTP and SFTP was encryption: “ftp is unencrypted, sftp is encrypted, but if you run everything on 'localhost' then it does not matter if you use one over the other” — <https://twitter.com/stigatle/status/397059080499789824> This ticket proposes to deprecate and then remove the LAFS-FTPd implementation in favor of LAFS-SFTPd. The justification is that LAFS-FTPd lacks important functionality, like mutable files, error reporting, and non-ASCII filenames, not to mention confidentiality and integrity, and we have no plans to add it, because the FTP protocol can't support some of those features, and because we've already implemented all of that in LAFS-SFTPd and we think anyone who uses LAFS-FTPd could (with only a *little* added effort) switch to LAFS-SFTPd. I'm marking this with the tag `forward-compatibility` and putting it into Milestone 1.11 because if we want to leave the deprecated LAFS-FTPd functionality in place for a full major release, then *not* doing the deprecation notice in 1.11 will obligate us to keep LAFS-FTPd functionality running in 1.12.

zooko added the

labels 2013-11-03 18:54:23 +00:00

zooko added this to the 1.11.0 milestone 2013-11-03 18:54:23 +00:00

daira commented

2013-11-03 21:55:00 +00:00

I don't think it would be all that much work to fix #680 and #1081. The problem is lack of developer resources; I would like to make the FTP implementation properly tested in the same way SFTP is, and I think that testing is as important to making FTP work well as fixing the bugs we currently know about.

PRabahy commented

2014-04-22 18:01:27 +00:00

I would consider #2213 a blocker to this from a usability standpoint. The description calls this out a reason that people use standard FTP instead, so #2213 removes that pain point.

warner added

code-frontend

and removed

unknown

labels 2014-09-02 17:26:50 +00:00

warner modified the milestone from 1.11.0 to 1.12.0

2014-09-02 17:26:50 +00:00

warner commented

2014-09-11 22:17:13 +00:00

I'm -0 on removing FTP. My main reason is that the FTP model of a filesystem is much much closer to Tahoe's own model: less "impedance mismatch", so I think it's going to give a much better experience. FTP works strictly in terms of PUT and GET, which tahoe does great.

SFTP has notions of read/write/seek/attrs, which are common for real (local) POSIX filesystems, but are hard to simulate (requiring temporary buffers until close(), like the various not-so-happy-making FUSE implementations). This fits better with mutable files, at the cost of complexity (sftpd.py is over 6x larger than ftpd.py), but I'm not convinced that most people use mutable files for general-purpose storage. It's true that SFTP is more modern and has important+useful properties (better unicode, file modes, directory management), but personally I'd expect better behavior from telling my OS to mount a localhost FTP server than a similar SFTP server.

That said, I understand how people will ignore docs and use whatever seems obvious, so if people are actively being hurt (made insecure) by the presence of the FTP server, and if the SFTP server is really a satisfactory replacement, then I'm willing to let go of FTP.

I'm -0 on removing FTP. My main reason is that the FTP model of a filesystem is much much closer to Tahoe's own model: less "impedance mismatch", so I think it's going to give a much better experience. FTP works strictly in terms of PUT and GET, which tahoe does great. SFTP has notions of read/write/seek/attrs, which are common for real (local) POSIX filesystems, but are hard to simulate (requiring temporary buffers until `close()`, like the various not-so-happy-making FUSE implementations). This fits better with mutable files, at the cost of complexity (sftpd.py is over 6x larger than ftpd.py), but I'm not convinced that most people use mutable files for general-purpose storage. It's true that SFTP is more modern and has important+useful properties (better unicode, file modes, directory management), but personally I'd expect better behavior from telling my OS to mount a localhost FTP server than a similar SFTP server. That said, I understand how people will ignore docs and use whatever seems obvious, so if people are actively being hurt (made insecure) by the presence of the FTP server, and if the SFTP server is really a satisfactory replacement, then I'm willing to let go of FTP.

warner added

code-frontend-ftp-sftp

and removed

code-frontend

labels 2014-12-02 19:50:38 +00:00

warner commented

2016-03-22 05:02:25 +00:00

Milestone renamed

warner modified the milestone from 1.12.0 to 1.13.0

2016-03-22 05:02:25 +00:00

warner commented

2016-06-28 18:17:14 +00:00