silent failure of bin/tahoe when root partition is out of space? #1279

New Issue

tahoe-lafs · 2010-12-12T03:56:24Z

davidsarah commented

2010-12-12 03:56:24 +00:00

Kiba on IRC reported that after following quickstart.html from a darcs checkout, bin/tahoe printed nothing for any command (including bin/tahoe --version-and-path). Our theory is that this is due to lack of space on the root partition. The output of python2 -v bin/tahoe --version-and-path included some warnings that .pyc files could not be generated (presumably due to lack of space).

Kiba on IRC reported that after following quickstart.html from a darcs checkout, `bin/tahoe` printed nothing for any command (including `bin/tahoe --version-and-path`). Our theory is that this is due to lack of space on the root partition. The output of `python2 -v bin/tahoe --version-and-path` included some warnings that .pyc files could not be generated (presumably due to lack of space).

tahoe-lafs added the

labels 2010-12-12 03:56:24 +00:00

tahoe-lafs added this to the undecided milestone 2010-12-12 03:56:24 +00:00

davidsarah commented

2010-12-12 03:59:49 +00:00

Attachment output.txt (7003 bytes) added

Output of 'python2 -v bin/tahoe --version-and-path' on Kiba's ArchLinux system

**Attachment** output.txt (7003 bytes) added Output of 'python2 -v bin/tahoe --version-and-path' on Kiba's [ArchLinux](wiki/ArchLinux) system

output.txt

6.8 KiB

tahoe-lafs added

1.8.1

and removed

1.8.0

labels 2010-12-12 04:00:31 +00:00

davidsarah commented

2011-01-06 03:38:09 +00:00

It's not clear whether Tahoe code is responsible, or one of our dependencies, or whether Python in general fails silently in low disk space situations.

The earliest we can print a warning without major contortions would be in source:src/allmydata/init.py. It would be best to import as little as possible before doing the check, which we could do by moving fileutil.get_disk_stats and get_available_space into their own file (since source:src/allmydata/util/fileutil.py imports code from twisted and pycryptopp). If we apply the patch on #1274, then we can check disk space without importing pywin32 on Windows.

OTOH, without knowing precisely what fails, we don't know which path to check. We could check the partition containing allmydata.*file*, and/or the one containing sys.executable, I suppose.

It's not clear whether Tahoe code is responsible, or one of our dependencies, or whether Python in general fails silently in low disk space situations. The earliest we can print a warning without major contortions would be in source:src/allmydata/*init*.py. It would be best to import as little as possible before doing the check, which we could do by moving `fileutil.get_disk_stats` and `get_available_space` into their own file (since source:src/allmydata/util/fileutil.py imports code from twisted and pycryptopp). If we apply the patch on #1274, then we can check disk space without importing pywin32 on Windows. OTOH, without knowing precisely what fails, we don't know which path to check. We could check the partition containing `allmydata.*file*`, and/or the one containing `sys.executable`, I suppose.

davidsarah commented

2011-01-06 03:42:21 +00:00

Replying to davidsarah:

It's not clear whether Tahoe code is responsible, or one of our dependencies, or whether Python in general fails silently in low disk space situations.

I should have read output.txt more carefully. It must be Python, since the failure occurs before any Tahoe code or dependencies have been imported.

That means we can't fix this in Tahoe.

Replying to [davidsarah](/tahoe-lafs/trac-2024-07-25/issues/1279#issuecomment-81519): > It's not clear whether Tahoe code is responsible, or one of our dependencies, or whether Python in general fails silently in low disk space situations. I should have read [output.txt](/tahoe-lafs/trac-2024-07-25/attachments/000078ac-9fd2-1ec5-9a17-adbe6c7d3c0d) more carefully. It must be Python, since the failure occurs before any Tahoe code or dependencies have been imported. That means we can't fix this in Tahoe.

tahoe-lafs added the

wontfix

label 2011-01-06 03:42:21 +00:00

davidsarah closed this issue

2011-01-06 03:42:21 +00:00

zooko commented

2011-01-06 06:58:41 +00:00

I guess the right thing to do is open a bug report with http://bugs.python.org and cross-link with this ticket...

I guess the right thing to do is open a bug report with <http://bugs.python.org> and cross-link with this ticket...

zooko commented

2011-01-17 07:52:38 +00:00

Actually, if the problem is triggered by python's failed attempt to write .pyc files, then perhaps we could work-around the problem by setting the blessed PYTHONDONTWRITEBYTECODE=1 environment variable, or passing the equivalent -B option to python (only available in python >= 2.6).

Kiba: could you please run your filesystem out of space again and reproduce this problem after setting PYTHONDONTWRITEBYTECODE=1?

We should consider using that setting for other reasons -- stale .pyc files that are left behind when their .py file has been mv'ed or rm'ed are a source of confusion, and the .pyc files can contain file locations which are stale because the .pyc and .py were both moved since the .pyc was generated. Also, the presence of .pyc's might encourage someone to delete or omit the corresponding .py's, thus making it harder for people to get access to the running source code. Finally, the time spent to write .pyc files is a performance penalty for startup. I wouldn't be surprised if PYTHONDONTWRITEBYTECODE=1 doesn't improve startup performance even when .pyc files are already present, too.

Actually, if the problem is triggered by `python`'s failed attempt to write `.pyc` files, then perhaps we could work-around the problem by setting the blessed `PYTHONDONTWRITEBYTECODE=1` environment variable, or passing the equivalent `-B` option to `python` (only available in `python >= 2.6`). Kiba: could you please run your filesystem out of space again and reproduce this problem after setting `PYTHONDONTWRITEBYTECODE=1`? We should consider using that setting for other reasons -- stale `.pyc` files that are left behind when their `.py` file has been mv'ed or rm'ed are a source of confusion, and the `.pyc` files can contain file locations which are stale because the `.pyc` and `.py` were both moved since the `.pyc` was generated. Also, the presence of `.pyc`'s might encourage someone to delete or omit the corresponding `.py`'s, thus making it harder for people to get access to the running source code. Finally, the time spent to write `.pyc` files is a performance penalty for startup. I wouldn't be surprised if `PYTHONDONTWRITEBYTECODE=1` doesn't improve startup performance even when `.pyc` files are already present, too.

zooko removed the

wontfix

label 2011-01-17 07:52:38 +00:00

zooko reopened this issue

2011-01-17 07:52:38 +00:00

zooko commented

2011-04-17 19:05:26 +00:00

See also #871 which is more about what happens when the disk fills up after the server has started.

Sign in to join this conversation.