Επιτροπή Συστημάτων ΤΜΗΥΠ

HP Ultrium LTO3 drive, 400/800GB
24 tapes changer
23 tapes + mailslot
23x400GB = 9.2TB net capacity
23x800GB = 18.4TB uncompressed capacity

25/6/2019:
Odometer 13963
Total power on time 68128h

Default service password: 42311324

new pass: 00000000

# lsscsi -g
[0:0:0:0]    cd/dvd  TEAC     DW-224E-R        C.AB  /dev/sr0   /dev/sg0 
[2:0:0:0]    disk    ATA      WDC WD1002FBYS-0 0C06  /dev/sda   /dev/sg3 
[3:0:0:0]    disk    ATA      WDC WD1002FBYS-0 0C06  /dev/sdb   /dev/sg4 
[4:0:0:0]    disk    ATA      WDC WD5000AAKS-0 3B01  /dev/sdc   /dev/sg5 
[5:0:0:0]    disk    ATA      WDC WD5000AAKS-0 3B01  /dev/sdd   /dev/sg6 
[7:0:4:0]    tape    HP       Ultrium 3-SCSI   G54W  /dev/st0   /dev/sg1 
[7:0:4:1]    mediumx HP       MSL G3 Series    2.40  /dev/sch0  /dev/sg2

tape drive control via mt tape changer control via mtx

load tape

# mtx -f /dev/sg2 load <tape #>

unload tape (to previous slot)

# mtx -f /dev/sg2 unload

check inventory

# mtx -f /dev/sg2 status

rewind tape

# mt -f /dev/nst0 rewind

dump/restore, mt etc. use /dev/st0 or /dev/nst0 (no rewind)
mtx uses raw SCSI device /dev/sg2

write from local file

# dd if=root.fs of=/dev/nst0 bs=4M status=progress

dump from remote machine to tape, useful for multivolume backups

# export RSH=/usr/bin/ssh
# export RMT=/usr/sbin/rmt
# dump -0uaf <IP or hostname>:/dev/nst0 <filesystem>

stackexchange

Ok, I think I've worked this out. TL;DR

Use dd with a large block size to read from the tape instead:

dd if=/dev/nst0 bs=1M | tar tvf -

Background

When you write to tapes, the data is written in units called blocks. These are like sectors on a hard disk. Where hard disk blocks were fixed at 512-bytes for many years and only recently moved to 4096-byte blocks, tape blocks can be set to any size you like.

The block size you wish to use is set with the setblk subcommand in mt-st:

mt-st -f /dev/nst0 setblk 512 # Use 512-byte blocks mt-st -f /dev/nst0 setblk 64k # Use 65536-byte blocks

When you issue a read operation to the drive, it will return data in block-sized chunks. You can't read half a block - the smallest amount of data you can read from a tape is one block, which of course could be any number of actual bytes depending on what the block size is.

This means if the program you are using supplies a 16kB memory buffer, you will be able to read up to 32 blocks at a time from the tape with 512-byte blocks as these fit exactly in the 16kB buffer. However you will not be able to read anything from the tape with 64kB blocks, because you can't fit even one of them into the 16kB buffer, and remember you can't read anything less than one whole block at a time.

Should you attempt to do this, by using a buffer that's too small for one block, the driver (in this case the st SCSI tape driver) will return a memory allocation error code to advise you that your read buffer is too small to hold even a single block.

To further complicate matters, some tape drives (apparently the LTO ones I am using) also support variable-sized blocks. This means the block size is determined by the size of each write operation and each block can be a different size to the last.

This mode is set with a block size of zero:

mt-st -f /dev/nst0 setblk 0 # Use variable-sized blocks

This is also the default option as - presumably, I am guessing here - it wastes less space with an incorrectly configured program. If, for example, you had set 4k blocks but your program only wrote data in units of 512 bytes at a time, there is a risk that each 512-byte chunk of data would take up 4k on the tape. Cause

If you now put everything together, you will realise that a tape can hypothetically have a 512-byte block followed by a 64kB block. If the program is reading the tape with a 16kB buffer, it will successfully read the first block, but then when it tries to read more, it won't be able to fit the following 64kB block in its buffer so the driver will return an error.

This explains why I was getting Cannot allocate memory errors most of the time, and occasionally I was able to get tar to extract the first few files but then I got the error again. I had not set the block size with mt-st so it had defaulted to variable-sized blocks when the tape was written, and now tar was using too small a buffer to read in some of those blocks.

tar has a couple of options for setting its own internal block sizes, namely –blocking-factor, –read-full-records, and –record-size, however these only work if tar is used to directly read and write to the tape.

Because I wrote to the tape through the mbuffer program to reduce tape shoe-shining, the block size in the tar archive no longer matched the block size on the tape. This meant –blocking-factor had little effect - it would allow the first block on the tape to be read, which includes a header telling tar what the blocking factor is supposed to be, wherein it switches to that and ignores the value given on the command line. This means the second and subsequent blocks can no longer be read! Solution

The solution is to use another program to read from the tape - one that can have the read buffer size set to a value large enough to hold the biggest block we are likely to see.

dd works for this, and in a pinch this works:

dd if=/dev/nst0 bs=256k | tar tvf -

You may need to increase 256k if your tape has larger blocks on it, but this worked for me. 1M also works fine so it doesn't appear to matter if the value is too large, within reason.

00.01.01 02:03:50.41 LIB/ERR <80 9B 28 00 > -- HE: sled position not found
00.01.01 02:00:44.09 LIB/ERR <80 9B 28 00 > -- HE: sled position not found
00.01.01 01:58:23.58 LIB/ERR <80 9B 28 00 > -- HE: sled position not found
00.01.01 01:55:18.21 LIB/ERR <80 9B 28 00 > -- HE: sled position not found
00.01.01 01:53:37.59 LIB/ERR <80 9B 28 00 > -- HE: sled position not found
00.01.01 01:49:45.68 LIB/ERR <80 9B 28 00 > -- HE: sled position not found
00.01.01 01:47:39.06 LIB/ERR <80 9B 28 00 > -- HE: sled position not found
00.01.01 01:39:17.14 LIB/ERR <80 9B 28 00 > -- HE: sled position not found
00.01.01 00:03:59.27 LIB/ERR <80 9B 28 00 > -- HE: sled position not found
14.09.03 16:07:15.95 LIB/ERR <80 9B 28 00 > -- HE: sled position not found
14.04.23 14:08:16.79 LIB/ERR <80 9B 28 00 > -- HE: sled position not found
13.05.07 13:48:54.39 LIB/ERR <80 9B 28 00 > -- HE: sled position not found
11.12.16 09:05:09.23 LIB/ERR <80 9B 28 00 > -- HE: sled position not found
11.05.27 14:00:59.32 LIB/ERR <80 8A 45 15 02 02 07 04 01 > -- HE: slider blocked
11.04.11 17:56:24.52 LIB/ERR <80 8A 45 15 02 02 07 04 01 > -- HE: slider blocked
10.09.27 15:21:51.43 LIB/ERR <80 9B 28 00 > -- HE: sled position not found
10.08.03 15:21:24.19 LIB/ERR <80 81 24 15 02 04 01 02 10 > -- HE: no response from BCR
10.08.03 15:04:02.77 LIB/ERR <80 81 24 00 > -- HE: no response from BCR
10.01.24 10:48:02.92 LIB/ERR <80 9B 28 00 > -- HE: sled position not found

19.07.05 19:08:20.47 TRC/CST <0A 0F 01
19.07.05 18:53:59.49 TRC/CST <0A 0F 01
19.07.05 17:45:21.22 TRC/CST <0A 0F 01
19.07.05 17:40:09.27 TRC/CST <01 02 00 02 0F 04 01
19.07.05 17:36:43.14 TRC/CST <01 02 00 04 01 02 0E
19.07.05 17:29:56.96 TRC/CST <0A 0F 01
19.07.05 17:17:30.66 TRC/CST <0A 0F 01
19.07.05 17:12:12.20 TRC/CST <00 00 00 00 00 13 00 00 20 FF FF FF FF 00 00 00 00 68 E6 41 20 F0 CE 43 20 E8 49 44 20 E8 49 44
19.07.05 16:51:55.05 TRC/CST <0A 0F 01
19.07.05 16:17:27.84 TRC/CST <00 00 00 00 00 13 00 00 20 FF FF FF FF 00 00 00 00 68 E6 41 20 F0 CE 43 20 E8 49 44 20 E8 49 44
19.07.05 16:17:27.76 TRC/CST <07 02
19.07.05 16:16:13.66 TRC/CST <06 02
19.07.05 16:14:01.26 TRC/CST <00 00 00 00 00 13 00 00 20 FF FF FF FF 00 00 00 00 AC D0 43 20 F0 CE 43 20 E8 49 44 20 E8 49 44
19.07.05 16:10:55.37 TRC/CST <00 00 00 00 00 13 00 00 20 FF FF FF FF 00 00 00 00 00 00 00 00 00 00 5D 04 07 00 00 00 07 00 00

19.07.05 19:08:20.47 TRC/CST <0A 0F 01
19.07.05 18:53:59.49 TRC/CST <0A 0F 01
19.07.05 17:45:21.22 TRC/CST <0A 0F 01
19.07.05 17:40:09.27 TRC/CST <01 02 00 02 0F 04 01
19.07.05 17:36:43.14 TRC/CST <01 02 00 04 01 02 0E
19.07.05 17:29:56.96 TRC/CST <0A 0F 01
19.07.05 17:17:30.66 TRC/CST <0A 0F 01
19.07.05 17:12:12.20 TRC/CST <00 00 00 00 00 13 00 00 20 FF FF FF FF 00 00 00 00 68 E6 41 20 F0 CE 43 20 E8 49 44 20 E8 49 44
19.07.05 16:51:55.05 TRC/CST <0A 0F 01
19.07.05 16:17:27.84 TRC/CST <00 00 00 00 00 13 00 00 20 FF FF FF FF 00 00 00 00 68 E6 41 20 F0 CE 43 20 E8 49 44 20 E8 49 44
19.07.05 16:17:27.76 TRC/CST <07 02
19.07.05 16:16:13.66 TRC/CST <06 02
19.07.05 16:14:01.26 TRC/CST <00 00 00 00 00 13 00 00 20 FF FF FF FF 00 00 00 00 AC D0 43 20 F0 CE 43 20 E8 49 44 20 E8 49 44
19.07.05 16:10:55.37 TRC/CST <00 00 00 00 00 13 00 00 20 FF FF FF FF 00 00 00 00 00 00 00 00 00 00 5D 04 07 00 00 00 07 00 00 

19.07.05 16:04:30.27 TRC/CST <4C 06 01
19.07.05 15:56:42.75 TRC/CST <4C 0F 00
19.07.05 15:53:50.70 TRC/CST <4C 0F 01
19.07.02 15:30:36.06 TRC/CST <4E 0F
00.01.01 02:27:59.67 TRC/CST <48 02 00 80
00.01.01 02:22:20.74 TRC/CST <48 02 00 C0
00.01.01 00:49:04.10 TRC/CST <48 02 00 80
00.01.01 00:00:07.06 TRC/CST <48 02 00 C0
00.01.03 19:44:57.86 TRC/CST <48 02 00 80
00.01.01 00:00:08.06 TRC/CST <48 02 00 C0
00.01.01 00:22:30.06 TRC/CST <48 02 00 80
00.01.01 00:00:08.06 TRC/CST <48 02 00 C0
00.01.01 00:20:33.71 TRC/CST <48 02 00 80
00.01.01 00:00:08.06 TRC/CST <48 02 00 C0
00.01.01 00:14:35.10 TRC/CST <48 02 00 80
00.01.01 00:00:08.06 TRC/CST <48 02 00 C0
00.01.01 00:29:56.36 TRC/CST <48 02 00 80
19.06.25 20:49:27.06 TRC/CST <4E 0F
00.01.01 00:09:00.07 TRC/CST <48 00 00 C0
00.01.01 00:08:57.56 TRC/CST <50 0F 00
19.06.25 19:28:43.06 TRC/CST <4E 0F