Bug 154930 - [zfs] cannot delete/unlink file from full volume -> ENOSPC
Summary: [zfs] cannot delete/unlink file from full volume -> ENOSPC
Status: Closed Not A Bug
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: 8.2-PRERELEASE
Hardware: Any Any
: Normal Affects Only Me
Assignee: freebsd-fs (Nobody)
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2011-02-21 09:30 UTC by Matthias Andree
Modified: 2015-07-29 08:40 UTC (History)
1 user (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Matthias Andree freebsd_committer freebsd_triage 2011-02-21 09:30:08 UTC
On a 100% full ZFS volume, deleting files is not possible and results
in ENOSPC (no space on device).  I suppose this is due to the write-ahead
logging nature.

Workaround: truncate files to be deleted to make room for the logs.

Suggestion: consider implicitly truncating a file to be deleted if there
is not sufficient disk space to delete the file otherwise.
Comment 1 Mark Linimon freebsd_committer freebsd_triage 2011-02-22 03:26:44 UTC
Responsible Changed
From-To: freebsd-ports-bugs->freebsd-fs

Does not appear to be a ports PR.  Reclassify.
Comment 2 Martin Matuska freebsd_committer freebsd_triage 2011-02-22 14:30:34 UTC
I was unable to reproduce your problem.

But I was able to reproduce a different situation:
- on a dataset with one or more snapshots I am unable to delete files
(ENOSPC) if the dataset got full.

If this is your case, then:
- deleting files does not unlink them from the snapshot.
- you must first delete a specific snapshot (or all snapshots linking
the file) to free space.

Cheers,
mm
Comment 3 Matthias Andree freebsd_committer freebsd_triage 2011-02-22 15:06:47 UTC
Am 22.02.2011 15:30, schrieb Martin Matuska:
> I was unable to reproduce your problem.
>
> But I was able to reproduce a different situation:
> - on a dataset with one or more snapshots I am unable to delete files
> (ENOSPC) if the dataset got full.
>
> If this is your case, then:
> - deleting files does not unlink them from the snapshot.
> - you must first delete a specific snapshot (or all snapshots linking
> the file) to free space.

Hi Martin,

no snapshots were ever used on the zpools or zfs volumes -- I had 
checked that previously. Only truncation of a 20 M file would allow me 
to delete files.

Best regards
Matthias

-- 
Matthias Andree
ports committer
Comment 4 soralx 2011-06-19 14:54:26 UTC
All,

 I encountered a similar snag, only worse, with no solution.

 The server has a 7 TB ZFS volume, which consists of 5 WD2001FASS
 connected to PERC 5/i in RAID5 (originally there were 4 disks, but the
 RAID was expanded by adding another disk and rebuilding). On June 12th
 (the date of OS rebuild, both world and kernel; the last update was on
 June 6), the pool was at v15, and had 12 GB free. On June 14, the pool
 already had 0 bytes free (found out from periodic daily run output
 email). It is unknown whether the FS was accessed at all during these
 two days; however, `find ./ -newerBt 2011-06-10 -print` returns just
 one small file dated 2011-06-11, and `find ./ -newermt 2011-06-10`
 returns ~20 files with total size <1 GB (BTW, these are backups, and
 nobody could have touched them, so it's a mystery why they have a
 recent modified time). So, the question is: where could have 12 GB
 suddenly disappear to?

 Further, on June 18th, the pool was mounted when a `zpool upgrade tst`
 command was issued. The upgrade to v28 succeeded, but then I found out
 that deleting files, large or small, is impossible:

# rm ./qemu0.raw
rm: ./qemu0.raw: No space left on device
# truncate -s0 ./qemu0.raw
truncate: ./qemu0.raw: No space left on device
# cat /dev/null > ./qemu0.raw
./qemu0.raw: No space left on device.

 Snapshots, compression, or deduplication have never been used on the volume.

 Also, these messages appear in dmesg:
Solaris: WARNING: metaslab_free_dva(): bad DVA 0:5978620460544
Solaris: WARNING: metaslab_free_dva(): bad DVA 0:5978620461568
Solaris: WARNING: metaslab_free_dva(): bad DVA 0:5993168926208

 Contrary to what the pool's name might suggest, this is not test storage,
 but has valuable data on it. Help!

>Environment:
System: FreeBSD cydem.org 8.2-STABLE FreeBSD 8.2-STABLE #0: Sun Jun 12 07:55:32 PDT 2011 soralx@cydem.org:/usr/obj/usr/src/sys/CYDEM amd64

`df`:
Filesystem    1K-blocks        Used      Avail Capacity  Mounted on
tst           7650115271 7650115271          0   100%    /stor1-tst

`zpool list`:
NAME   SIZE  ALLOC   FREE    CAP  DEDUP  HEALTH  ALTROOT
tst   5.44T  5.36T  80.6G    98%  1.00x  ONLINE  -

`zfs list`:
NAME   USED  AVAIL  REFER  MOUNTPOINT
tst   7.12T      0  7.12T  /stor1-tst

`zpool status`: 
  pool: tst
 state: ONLINE
 scan: scrub in progress since Sat Jun 18 06:32:37 2011
    1.82T scanned out of 5.36T at 350M/s, 2h56m to go
    0 repaired, 34.02% done
config:

        NAME        STATE     READ WRITE CKSUM
        tst         ONLINE       0     0     0
          mfid1     ONLINE       0     0     0

errors: No known data errors
[this completed with 'scrub repaired 0 in 5h47m with 0 errors' at 133% done,
 i.e. it scrubbed all 7.12 TB].


-- 
[SorAlx]  ridin' VN2000 Classic LT
Comment 5 commit-hook freebsd_committer freebsd_triage 2015-07-28 22:49:41 UTC
A commit references this bug:

Author: bdrewery
Date: Tue Jul 28 22:48:59 UTC 2015
New revision: 285990
URL: https://svnweb.freebsd.org/changeset/base/285990

Log:
  unlink(2): Note the possibility for ENOSPC to be returned on ZFS.

  PR:		154930

Changes:
  head/lib/libc/sys/unlink.2
Comment 6 Steven Hartland freebsd_committer freebsd_triage 2015-07-29 08:40:05 UTC
In addition later ZFS versions reserve more privileged space for this very reason see:
https://svnweb.freebsd.org/base?view=revision&revision=268473