Bug 230493 - sysutils/dvd+rw-tools: Cannot append data to BD-R DL used more than 25gb
Summary: sysutils/dvd+rw-tools: Cannot append data to BD-R DL used more than 25gb
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Only Me
Assignee: freebsd-ports-bugs (Nobody)
URL:
Keywords: easy
Depends on:
Blocks:
 
Reported: 2018-08-10 02:20 UTC by Masachika ISHIZUKA
Modified: 2018-08-21 07:19 UTC (History)
4 users (show)

See Also:
ish: maintainer-feedback+
koobs: merge-quarterly+


Attachments
patch from debian Bug#615978 (591 bytes, patch)
2018-08-10 02:20 UTC, Masachika ISHIZUKA
koobs: maintainer-approval+
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Masachika ISHIZUKA 2018-08-10 02:20:53 UTC
Created attachment 196043 [details]
patch from debian Bug#615978

I want to append data to DB-R DL (50GB disc) as follows.

# growisofs -udf -iso-level 3 -M /dev/cd0 -J -R --graft-points /foo/=/foo2
Executing 'mkisofs -udf -iso-level 4 -C 16,16914560 -M /dev/fd/0 -J -R --graft-points /foo/=/foo2 | builtin_dd of=/dev/pass1 obs=32k seek=1057160'
Warning: Creating ISO-9660:1999 (version 2) filesystem.
Warning: ISO-9660 filenames longer than 31 may cause buffer overflows in the OS.
Setting input-charset to 'UTF-8' from locale.
ISO-9660 image includes checksum signature for correct inode numbers.
SUSP signatures version 1 found
Rock Ridge signatures version 1 found
Rock Ridge id 'RRIP_1991A'
:-( /dev/pass1: -9521408 blocks are free, 7252777 to be written!
:-( write failed: No space left on device

This bug was fixed by debian Bug#615978: dvd+rw-tools: While burning ISO Multisession BD-R 50G growisofs reports no space left while 24G+ are available.

After applying patch, I can append data to DB-R DL.
Comment 1 Kubilay Kocak freebsd_committer freebsd_triage 2018-08-10 03:37:15 UTC
Comment on attachment 196043 [details]
patch from debian Bug#615978

Approved by: portmgr (unmaintained port)

Needs/Pending QA
Comment 2 Kubilay Kocak freebsd_committer freebsd_triage 2018-08-10 03:57:43 UTC
This change appears to address the same (?) issue as that addressed in ports r383719 a number of years ago committed by danfe (CC'd), but in a different way; adding 'bdr_plus_pow -1' rather than removing the '&& !bdr_plus_pow' conditional. 

That change came from an Ubuntu bug report [2], which references a different Debian bug [3].

It would be good to clarify if/why this issue is still reproducible in the port with the existing patch, before committing what appears (my lack of expertise of C code not withstanding) to be a noop.

[1] https://svnweb.freebsd.org/changeset/ports/383719
[2] https://bugs.launchpad.net/ubuntu/+source/dvd+rw-tools/+bug/1113679/
[3] http://bugs.debian.org/713016

Since this port has no maintainer, request feedback from the reporter
Comment 3 Kubilay Kocak freebsd_committer freebsd_triage 2018-08-10 03:58:34 UTC
(In reply to Kubilay Kocak from comment #2)

"bdr_plus_pow -1" above should be "bdr_plus_pow = 1"
Comment 4 Masachika ISHIZUKA 2018-08-10 05:04:03 UTC
(In reply to Kubilay Kocak from comment #3)

I checked dvd+rw-tools on debian 7.1-12.
This applied 12 patches.

debian/patches/10-blue-ray-bug713016.patch: Closes: #713016
debian/patches/ignore_pseudo_overwrite.patch: Closes: #615978
(Other patches are omitted.)

These patches were different and I think they shoud be applied.
Comment 5 scdbackup 2018-08-10 08:03:18 UTC
Hi,

ports r383719 / Ubuntu 1113679 is about a differnt bug, which caused
an inappropriate command to be sent to the drive and consequentially
an error reoly by the drive and an error message by growisofs.
The mistake was that the BD-R was formatted automatically but growisofs
still memorized the unformatted state.

About the bug discussed here (Debian 615978):

            nwa = 0;
            if (buf[7]&1 && !bdr_plus_pow)      // NWA_V
            {   nwa  = buf[12]<<24;
                nwa |= buf[13]<<16;
                nwa |= buf[14]<<8;
                nwa |= buf[15];
            }
            free_blocks  = buf[16]<<24;
            free_blocks |= buf[17]<<16;
            free_blocks |= buf[18]<<8;
            free_blocks |= buf[19];
            ret = nwa + free_blocks;

This obviously is meant to compute the total capacity of the medium,
as sum of the effectively consumed blocks (Next Writable Address, NWA)
and of the free blocks.
Andy Polyakov expected the whole media capacity to be reported by
the Free Blocks field of the drive's reply, because the BD-R is in
state Pseudo-Overwrite, where all addresses are valid for writing.
But the fact that the NWA_V bit in buf[7] is set announces a valid
NWA and thus indicates that the drive does not count all blocks as free.

---------------------------------------------------------------------

The only riddle that remains is why this bug is reported only with
BD-R DL. MMC-5 4.5.3.6.4 gives an example which does not match the
code neither with the fix nor without it. But the example situation
differs somewhat from growisofs' doings. If i mentally move it towards
growisofs, then the fixed code seems more plausible.

All in all Pseudo-Overwrite is complicated and based on the mechanisms
of Defect Management, which do not work very well when actually needed.

I have a single-layer POW BD-R for testing purposes (so libburn-1.4.8
can refuse to write onto it).
Indeed the drive reports to libburn for its "invisible" track a
track description which would match the unfixed code:

  $ xorriso -scsi_log on -outdev /dev/cd0
  ...
  READ TRACK INFORMATION
  52 01 00 00 00 01 00 00 14 00 
  From drive: 20b
  00 26 01 01 00 04 61 01 00 00 00 00 00 00 00 00 00 b4 74 00
       5783 us     [ 5021775 ]
  ...

buf[7] is 0x01 here, NWA is 0x00000000, Free Blocks is 0x00b47400
= 11826176, which matches READ CAPACITY 0x00b473ff = 11826175.

So for this medium, the unfixed code would work.
The fixed code would work too, because NWA is 0.

I failed yet to read from MMC specs that BD-R DL shall bevave different
from BD-R.
The statement of growisofs:
  builtin_dd of=/dev/pass1 obs=32k seek=1057160
indicates that the burn run was to start at block 16*1057160 =
16,914,560 which is already in the second layer.


Have a nice day :)

Thomas
Comment 6 Kubilay Kocak freebsd_committer freebsd_triage 2018-08-10 09:23:06 UTC
(In reply to scdbackup from comment #5)
.
Thank you for the detail Thomas, and your reply clarifying too Masachika

I should have checked this earlier, but it appears the patch (bdr_plus_pow = 1;) from ports r383719 is set later (line 756) in the source file, than the patch (&& !bdr_plus_pow removal) here, line 410. 

I guess that doesn't make it (this patch) a noop, which was my original question.
Comment 7 Kubilay Kocak freebsd_committer freebsd_triage 2018-08-10 09:24:24 UTC
Bugzilla wont let me set https urls, ok. 

Apologies for the noise.
Comment 8 scdbackup 2018-08-10 10:05:25 UTC
Hi,

> I guess that doesn't make it (this patch) a noop, which was my original
> question.

Both patches are appropriate and needed.

---------------------------------------------------------------------

I am still puzzled what it may mean that the POW BD-R DL is offered by
the drive with non-zero NWA.

@Masachika ISHIZUKA:

Do you see the added files of the appended session if you mount
the BD-R medium with option -s 0 ?

  mount_cd9660 -o noexec,nosuid -s 0 /dev/cd0 $YOUR_MOUNT_POINT

Reasoning:

According to Andy Polykov about 10 years ago, the main reason for
using POW was that old Solaris versions do not obey the normal rule
for multi-session to mount the first track of the last session on
medium. growisofs works around this shortcomming by writing a
ISO 9660 superblock copy of the last track begining at block 0.
This is on BD-R possible only with POW. The medium shall look to the
reader as if it had only one track. (I.e. like DVD+RW or BD-RE.)

BD-R + POW is classified by growisofs_mmc.cpp:poor_man_rewritable()
as overwritable medium. This should trigger the copying procedure in
growisofs.c:main() 

    if (next_session!=0 &&
        (!poor_man || poor_man_rewritable (ioctl_handle,the_buffer)))
    { ...

If the announcement of the drive is to be taken for serious, then this
copying of the superblock should fail. But Masachika ISHIZUKA reports
that it does not fail.

If for some reason the copying was omitted, then Masachika should see
only the files of the first session if mount option -s 0 is used.
If he sees all files of all sessions, then the copying succeeded
despite the NWA announcement of the drive.


Have a nice day :)

Thomas
Comment 9 commit-hook freebsd_committer freebsd_triage 2018-08-10 10:06:13 UTC
A commit references this bug:

Author: koobs
Date: Fri Aug 10 10:05:20 UTC 2018
New revision: 476826
URL: https://svnweb.freebsd.org/changeset/ports/476826

Log:
  sysutils/dvd+rw-tools: Fix issue appending more than 25gb on BD-R DL discs

  While burning ISO Multisession BD-R 50G growisofs reports no space left while
  24G+ are available.

  Backport a fix from Debian [1] to address the issue.

  While I'm here, document description and origins of both patches [1][2] that
  apply to same source file, for our future selves, and to avoid confusion
  as both chunks reference the same variable.

  [1] https://bugs.debian.org/615978
  [2] https://bugs.debian.org/713016

  PR:		230493
  Submitted by:	Masachika ISHIZUKA <ish amail plala or jp>
  Approved by:	portmgr (implicit, unmaintained port)
  Obtained from:	Debian [1]
  MFH:		2018Q3

Changes:
  head/sysutils/dvd+rw-tools/Makefile
  head/sysutils/dvd+rw-tools/files/patch-Makefile.m4
  head/sysutils/dvd+rw-tools/files/patch-growisofs__mmc.cpp
  head/sysutils/dvd+rw-tools/files/patch-mp.h
  head/sysutils/dvd+rw-tools/files/patch-transport.hxx
Comment 10 Kubilay Kocak freebsd_committer freebsd_triage 2018-08-10 10:09:50 UTC
(In reply to scdbackup from comment #8)

It would be very nice if one or both of you could maintain this port, given the clear experience/knowledge/background you both have. I'd be happy to help support you all/both bring it back up to scratch where its needed, if other patches are missing, etc.

The change has been committed, now pending MFH approval to the quarterly branch.
Comment 11 scdbackup 2018-08-10 10:29:18 UTC
Hi,

> It would be very nice if one or both of you could maintain this port,
> given the clear experience/knowledge/background you both have.

Well, my FreeBSD by which i make regression tests for libburn is still
version 8. Changing the test machine with its various antique systems
is not trivial at all.

But i offer my support for problems related to optical drives or to
the ISO 9660 filesystem (not about hybrids with UDF or HFS).
If you deem a PR related to my topics, then be invided to give me
a heads-up.

On a website which allows googlebotting i should get aware on my own.
I came to this PR by my daily Google patrol for "xorriso", "libburn",
"libisofs", "libisoburn", "cdrskin", "Xfburn", "Brasero", "growisofs".
(The first five being by me, the other three being orphaned, currently.)


Have a nice day :)

Thomas
Comment 12 Masachika ISHIZUKA 2018-08-10 10:48:58 UTC
(In reply to scdbackup from comment #8)

> Do you see the added files of the appended session if you mount
> the BD-R medium with option -s 0 ?
>
>  mount_cd9660 -o noexec,nosuid -s 0 /dev/cd0 $YOUR_MOUNT_POINT

Dispite specifying '-s 0', I see all directories/files of all sessions.
Comment 13 scdbackup 2018-08-10 11:10:02 UTC
Hi,

Masachika ISHIZUKA wrote:
> Dispite specifying '-s 0', I see all directories/files of all sessions.

Then the patch seems to be fully successful.

Have a nice day :)

Thomas
Comment 14 Masachika ISHIZUKA 2018-08-10 11:21:33 UTC
(In reply to Kubilay Kocak from comment #6)

> I should have checked this earlier, but it appears the patch (bdr_plus_pow = 1;) from ports r383719 is set later (line 756) in the source file, than the patch (&& !bdr_plus_pow removal) here, line 410. 
> I guess that doesn't make it (this patch) a noop, which was my original question.

I'm sorry.
Since the patch command is very clever, even if the order of patched lines is changed, it will work properly, but it is certainly better to fix it in ascending order. Excuse me.
Comment 15 commit-hook freebsd_committer freebsd_triage 2018-08-21 07:18:41 UTC
A commit references this bug:

Author: koobs
Date: Tue Aug 21 07:18:29 UTC 2018
New revision: 477708
URL: https://svnweb.freebsd.org/changeset/ports/477708

Log:
  MFH: r476826 sysutils/dvd+rw-tools: Fix issue appending more than 25gb on BD-R DL discs

  While burning ISO Multisession BD-R 50G growisofs reports no space left while
  24G+ are available.

  Backport a fix from Debian [1] to address the issue.

  While I'm here, document description and origins of both patches [1][2] that
  apply to same source file, for our future selves, and to avoid confusion
  as both chunks reference the same variable.

  [1] https://bugs.debian.org/615978
  [2] https://bugs.debian.org/713016

  PR:		230493
  Submitted by:	Masachika ISHIZUKA <ish amail plala or jp>
  Approved by:	portmgr (implicit, unmaintained port)
  Obtained from:	Debian [1]

  Approved by:	miwi (ports-secteam)

Changes:
_U  branches/2018Q3/
  branches/2018Q3/sysutils/dvd+rw-tools/Makefile
  branches/2018Q3/sysutils/dvd+rw-tools/files/patch-Makefile.m4
  branches/2018Q3/sysutils/dvd+rw-tools/files/patch-growisofs__mmc.cpp
  branches/2018Q3/sysutils/dvd+rw-tools/files/patch-mp.h
  branches/2018Q3/sysutils/dvd+rw-tools/files/patch-transport.hxx