Trac file uploads/attachments failing/corrupted #1581
Labels
No Label
0.2.0
0.3.0
0.4.0
0.5.0
0.5.1
0.6.0
0.6.1
0.7.0
0.8.0
0.9.0
1.0.0
1.1.0
1.10.0
1.10.1
1.10.2
1.10a2
1.11.0
1.12.0
1.12.1
1.13.0
1.14.0
1.15.0
1.15.1
1.2.0
1.3.0
1.4.1
1.5.0
1.6.0
1.6.1
1.7.0
1.7.1
1.7β
1.8.0
1.8.1
1.8.2
1.8.3
1.8β
1.9.0
1.9.0-s3branch
1.9.0a1
1.9.0a2
1.9.0b1
1.9.1
1.9.2
1.9.2a1
LeastAuthority.com automation
blocker
cannot reproduce
cloud-branch
code
code-dirnodes
code-encoding
code-frontend
code-frontend-cli
code-frontend-ftp-sftp
code-frontend-magic-folder
code-frontend-web
code-mutable
code-network
code-nodeadmin
code-peerselection
code-storage
contrib
critical
defect
dev-infrastructure
documentation
duplicate
enhancement
fixed
invalid
major
minor
n/a
normal
operational
packaging
somebody else's problem
supercritical
task
trivial
unknown
was already fixed
website
wontfix
worksforme
No Milestone
No Assignees
4 Participants
Notifications
Due Date
No due date set.
Reference: tahoe-lafs/trac-2024-07-25#1581
Loading…
Reference in New Issue
No description provided.
Delete Branch "%!s(<nil>)"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Since the migration (and switching from Apache/mod_python to nginx/uwsgi), uploads of moderate-sized files are failing in weird ways. Small patches sometimes work, but larger patches fail in weird ways:
Somebody needs to study the way that nginx invokes uwsgi with large request bodies. If it writes them to a file before handing them to uwsgi (seems unlikely), then it's conceivable that a file-permission/ownership problem is involved.
I'm going to assign this one to Zooko, since he configured nginx/uwsgi on this box, and thus far is the only one who even begins to understand them.
For now, I'm manually scp'ing files to tahoe-lafs.org and using 'trac-admin' to attach them to tickets.
The UnicodeDecodeError looks like it's coming from a part of Trac that
handles form POST arguments,
What I think is happening here is that file uploads use MIME-Multipart,
so each body section can have a couple of attributes in addition to the
actual value. "name" is one such attribute, as is "filename". I'm
guessing that the Trac code assumes that any attachment that doesn't
have a filename must be a UTF8-encoded unicode string. If that's the
case, then it implies that Trac is not seeing a
filename=
attribute on the main file's MIME-Multipart body. This could be a result
of nginx/uwsgi removing it, or something odd happening in my browser
such that it's not sending it.
Also, .jpg attachments (manually added with trac-admin) larger than 1MB don't render as images when you look at them: the message says
HTML preview not available, since the file size exceeds 1000000 bytes. Try downloading the file instead.
. I increased the preview limit to 1.2MB to accomodate this, which removes the error, but now it shows me the first inch or so of the image and then nothing else: it looks like the server stops serving image data after some limited-size buffer. This feels like another nginx/uwsgi issue.I was able to attach and download a file of size 1 MB just now. Now trying 10 MB...
Okay, I attempted to upload a 10 MB file and got an error from nginx. Then I changed nginx's config to allow client request bodies up to 2 GB and tried again. Now I got the error that Brian said Kevan had -- the upload appeared to succeed but then the file wasn't visible in the attachments list. Will investigate more later.
#1577 was a duplicate of this.
Attachment 10Mnull (10000000 bytes) added
Attachment 10Mnull.2 (10000000 bytes) added
Attachment 10Mnull.3 (10000000 bytes) added
Attachment 10Mnull.4 (10000000 bytes) added
I still see this occasionally, and not always with large files.
still seeing this, on a 33kB patch upload
it's worse, I just uploaded a 150kB image to #1265 and it managed to corrupt the file, by inserting a single newline in front of the whole file (not a text-mode-FTP -style problem, just a single-byte 0x0a prefix).
Trac file uploads/attachments failingto Trac file uploads/attachments failing/corruptedI don't know if it's related, but I just tried to attach a 43kB patch to #166 and got an "internal trac error" with the following exception in the logs:
My vague hunch is that the accept-an-attachment handler got called but somehow the attachment itself got lost on the way in.
One other data point, I re-tried attaching that same 43kB patch to #166, and got the previously-buggy "lalala I can't hear you" behavior, where I land on the expected "list of attachments" page but the file I just tried to upload wasn't listed. I repeated that several times with the same results.
Then, about 10 minutes later, I tried again, and it worked. Two things had happened in the meantime: Zooko restarted nginx "after changing its ssl cache size from 1m to 100m entries", and I changed the attachment comments (removed a few words).
At least one problem (maybe the only remaining one) does seem to be related to the description length. Attaching https://tahoe-lafs.org/trac/tahoe-lafs/attachment/ticket/1636/fix-1636-for-1.9.2.darcs.patch to #1636 with the following description failed:
as did the same description without the last sentence. The next attempt with the same file and this description worked:
Looks like this issue still exists. I tried to upload a .diff (571 bytes) to #1842, it looked like it succeeded, but when I refreshed the ticket there was no attachment. I waited about an hour and tried again with the same result.
Thanks for the update, PRabahy. I'll try to reproduce it using a .diff of 571 bytes length.
Possibly related: #2241
Possibly a duplicate: #2270
Ticket retargeted after milestone closed