--- dvbackup.sgml.orig 1970-01-01 03:00:00 +0300 +++ dvbackup.sgml 2004-04-11 16:29:04 +0400 @@ -0,0 +1,184 @@ +Robert"> + Jordens"> + Feb 19, 2004"> + 1"> + jordens@debian.org"> + + dvbackup"> + + Debian"> + GNU"> +]> + + + +
+ &dhemail; +
+ + &dhfirstname; + &dhsurname; + + + 2004 + &dhusername; + + &dhdate; + + + &dhucpackage; + + &dhsection; + + + &dhpackage; + + Converter from arbitrary data to a DV stream + + + + + &dhpackage; + + + + + + + + + + + + + + + + + + + DESCRIPTION + + This manual page documents briefly the + &dhpackage; tool. + + This manual page was written for the &debian; distribution + because the original program does not have a manual page. + + + As you probably know, current digital + camcorders can save approximately 13 GB of data on those tiny DV + cartridges at a speed of 3.6 MB/second. That's fast. Very fast. It's + faster than most DAT streamers which only work at 1 MB/sec or less. We + can not use all of the data, but 10 GB should be good enough for + everyone. + + That's nice, but how can we use this to save data on it? And here comes + the fun part: If you read the DV documentation carefully, you will + notice that the AC DCT coefficients of the video data blocks (8x8 + pixels in size) get a fixed amount of space in the DV data stream, but + can be terminated earlier with a certain code sequence. So let's have + some fun: We terminate the AC coefficients immediately leaving only the + DC coefficient for a fancy penguin picture and use the rest for our + backup data. Future implementations could easily add a little picture + showing the currently written file or something like that. + + Then there is the audio data, which is written uncompressed onto the + tape. That means: We tell the camcorder at the beginning of each frame, + that we won't use audio at all but fill the space reserved for it with + data. Easy, but somewhat hacky. In fact, I don't know, if this works on + every camcorder and not only on mine (a Sony VX700). Your mileage may + vary. + + To finally bring the data on tape, you have to use an additional + utility, called dvconnect, which is (hopefully soon) included into + libdv. Take a look at the patch manager if it's not in already. And + then it's time to rock and roll: + +Advantages of dvbackup over other backup technologies + + + relatively cheap (the cheapest camcorder will be enough, but if you + have already one...) + + the tapes are quite cheap + + open standard: if your streamer, aah camcorder dies you can rescue + your data with any other one (except PAL/NTSC need to fit), you are + not bound to a special company + + it's faster than many streamers and it will be more comfortable - + you can use the search-index function to "jump" to a recording + + tapes (re)wind faster than many streamers + + you do not need to rewind the tape to eject it + + + +Disadvantages of dvbackup + + you do not get any warranty :-) + + +Usage of the Unix client + + Press record on your camcorder. (Or use your favorite avc control + program for this. For the VX700 this doesn't work and you have to + hack something together, that uses LANC. I might publish my + "solution" for this soon...) + + Type "find . |cpio -o -H crc |dvbackup --prefix=125 |dvconnect -s" + to stream directly to your camcorder. This most likely does only + work on very fast harddisks and filesystems. You might try + something like "find . |cpio -o -H crc |dvbackup --prefix=125 + |dvconnect -s -b 500" Alternatively, you can write the data in + several parts on tape. Just go experimenting, and mail me the + resulting backup scripts... + + Stop your camcorder and rewind. + + Now it's time to verify: Press play on tape ;-) + + Type "dvconnect |dvbackup -t" and watch for crc errors. The data + corruption bug mentioned for version 0.0.1 seems to be fixed so + there is no excuse in not using this little nifty program ;-) + + If you want to restore: Do a simple "dvconnect |dvbackup -d|cpio + -imV". CPIO will also happily tell you about CRC errors. So you + might want to check using cpio's archive test mode too. But keep in + mind, that cpio's CRC function is not that fast! + + + + + AUTHOR + This manual page was written by &dhusername; &dhemail; for the + &debian; system (but may be used by others). Permission is granted + to copy, distribute and/or modify this document under the terms of + the GNU General Public License, Version 2. On + Debian systems, the full text of this license can be found in the + file /usr/share/common-licenses/GPL-2. + + + + + + --- rsbep.sgml.orig 1970-01-01 03:00:00 +0300 +++ rsbep.sgml 2004-04-11 16:30:59 +0400 @@ -0,0 +1,222 @@ +Robert"> + Jordens"> + Feb 19, 2004"> + 1"> + jordens@debian.org"> + + rsbep"> + + Debian"> + GNU"> +]> + + + +
+ &dhemail; +
+ + &dhfirstname; + &dhsurname; + + + 2004 + &dhusername; + + &dhdate; + + + &dhucpackage; + + &dhsection; + + + &dhpackage; + + Reed-Solomon FEC (forward error correction) with + byte-spreading + + + + + &dhpackage; + + + + + + + + + + + DESCRIPTION + + This manual page documents briefly the + &dhpackage; tool. + + This manual page was written for the &debian; distribution + because the original program does not have a manual page. + + &dhpackage;is a tool that protects the data-stream + from stdin with Reed-Solomon FEC (forward error correction) and additional + it spreads the bytes of the resulting blocks out to give some protection + against burst errors (e.g from tape-recordings). + + The Reed-Solomon-code is from Phil Karn (see REAMDE.rsbep.RS32), and + it's a special version for i386 (and compatible) to get enough performance + for realtime encoding/decoding making it suitable for use with the tool + "dvbackup" - which requires 3,6MByte/sec of data to feed the + MiniDV-Camcorder. It is hardwired for (255,223)-RS. + + The burst error protection is done by encoding ERR_BURST_LEN blocks + of 255 bytes and placing the Bytes of an RS-block with distance + ERR_BURST_LEN withing a buffer of ERR_BURST_LEN*255 bytes size. (This is + the minimum output size of rsbep) + + During decoding the blocks are reassembled and then decoded by RS32. + If any error occured in the data it is corrected if possible, otherwise + written as is, the number of corrected failures and uncorrectable blocks is + printed if >0 or according to commandline options (v,q). + + The first line of each encoded output block contains + "rsbep 255 223 765 MAGIC_NUM" which is the identifier for + rsbep, the Reed-Solomon block-size, data-size and the ERR_BURST_LEN (that is + a number >=block-size and exact multiple of block-size). MAGIC_NUM is used + to resynchronize if the stream is completly broken, which makes it possible + to recover later files in your (tar-)archive. + + + + ERROR CORRECTION + + Now how much errors can it correct? + + RS(255,223) can correct up to 16 incorrect bytes in + unknown positions. + + However, the problem is, that typical communications and storage + errors are also burst errors - which is that a lot of subsequent bytes are + lost. That would mean, if more than 16 subsequent bytes are damaged, the + data are irreparably damaged. + + The method above spreads those bytes so far out, that up to + 16*255*3=12240 subsequent bytes can get garbled without loosing valuable data + at all! + + Test with dvbackup and the LongPlay mode of my camcorder showed that + more than enough to use it - it corrected Kilo-Bytes of errors in Giga-Bytes + of data. + + Of course there is a probability, that the failure has just that rate, + that it hit's always the bytes of the same block - i leave it to the + mathematicians among you to calculate it ;-) + + + + LIMITATIONS + + * It does'nt make use of "erasures" - that would double the recover + rate. + + * It is hardcoded for RS(255,223)*(<=255) bytes, if you change the + code you may not be able to recover old data with different + RS-values + + * The resynchronisation takes so much CPU that it will fillup the + buffer soon and anything will fall apart, (running the program 'offline' + might help) - it needs a lot of work to + be really of use... + + * Failures in the header-line might have drastic consequences, + mathematically spoken: Those bytes contain more information + + * The error correction works only from stream with failures, + not erasures, dvbackup has the option '-r' to take care of this to the + extend MiniDV allows. + + + + OPTIONS + + + + + + + Decode. The default is to encode. + + + + + + + Verbose. Talks about the errors it encounters and gives + statistics at the end. + + + + + + + Quiet. Output only a minimum of messages. + + + + + + + Override the default Reed-Solomon blocksize of 255 and use 'x' + instead. + + + + + + + Override the default Reed-Solomon datasize of 223 and use 'y' + instead. + + + + + + + Override the default ERR_BURST_LEN (used to spread the data) of + 765 and use 'z' instead. + + + + + + AUTHOR + This manual page was written by &dhusername; &dhemail; for the + &debian; system (but may be used by others). The text is based on + information from Guido Fiala. Permission is granted + to copy, distribute and/or modify this document under the terms of + the GNU General Public License, Version 2. On + Debian systems, the full text of this license can be found in the + file /usr/share/common-licenses/GPL-2. + + + + + + --- README.ALT.orig 1970-01-01 03:00:00 +0300 +++ README.ALT 2004-04-13 20:28:14 +0400 @@ -0,0 +1,48 @@ +README for ALTLinux package +--------------------------- + +We assume you have video1394 and raw1394 module loaded. +All examples for PAL camcoders. + +Make backup +----------- + +using RSBEP: +$ dvcont record; sleep 5; \ +find /what/dir | cpio -o -H crc | rsbep -v | \ +dvbackup -b 'title' -v -p 200 -o 200 | \ +dvconnect -s -v -u /usr/share/dvbackup/underrun-pal.dv; \ +dvcont stop + +not using RSBEP: +$ dvcont record; sleep 5; \ +find /what/dir | cpio -o -H crc | \ +dvbackup -b 'title' -v -p 200 -o 200 | \ +dvconnect -s -v -u /usr/share/dvbackup/underrun-pal.dv; \ +dvcont stop + +Verify backup +------------- + +using RSBEP: +$ dvcont play ; \ +dvconnect | dvbackup -d -v | rsbep -d -v | \ +cpio -i --only-verify-crc; dvcont stop + +not using RSBEP: +$ dvcont play ; \ +dvconnect | dvbackup -d -v | \ +cpio -i --only-verify-crc; dvcont stop + +Restore from backup +------------------- + +using RSBEP: +$ dvcont play ; \ +dvconnect | dvbackup -d -v | rsbep -d -v | \ +cpio -idm --no-absolute-filenames; dvcont stop + +not using RSBEP: +$ dvcont play ; \ +dvconnect | dvbackup -d -v | \ +cpio -idm --no-absolute-filenames; dvcont stop