# cddbd/cfg - "Build" settings for package #--------------------------------------------------------------------- [technotes] 1. This package compiles a "cddb" database server. It optionally builds a compatible copy of the FreeDB database, includ- ing a "fuzzy" index. As of 150613, the database "build" is suppressed by default due to high hardware requirements. It can be enabled, but the "build" isn't guaranteed to succeed. The "build" shouldn't be attempted unless the "build" system meets the requirements. For more information, see the comments later in this "cfg" file. #--------------------------------------------------------------------- 2. Most of the associated files are stored in a single XZM file loca- ted in the directory "/xzm". The XZM-file name (excluding path) is "cddbdata.xzm". The XZM file may be left in its default location or it may be moved to various other locations (see the script "xzm-boot-setup"). If it's in one of the supported locations at "boot" time, the package should op- erate correctly. For LiveDistro versions of the distro, it's a good idea to move the XZM file to a supported location outside the XZM space that the Live- Distro itself occupies. This will prevent nested XZM mounts (which are less efficient). #--------------------------------------------------------------------- 3. For this package to work correctly, the following changes must be made to the system and/or to other packages: (a) The following entry must be added to "/etc/hosts": 127.0.0.1 freedb.freedb.org (b) "sudo" must be configured so that ordinary users can run the scri- pt "cddbd-start" that's installed along with "cddbd". Specifically, the following entry should be added to "sudoers.in" in the "sudo" mas- ter resource tree. Subsequently, "sudo" should be rebuilt: ALL ALL = NOPASSWD: __META_PRODTREE__/cddbd/sbin/cddbd-start (c) Programs that use "freedb.freedb.org" must be modified so that they execute the command "cddbd-sudo" either at startup time or before they try to connect to "freedb.freedb.org". Note: The second approach is recommended. (d) Programs that use the HTTP interface to "freedb.freedb.org" (such as "xmms") will require additional changes. #--------------------------------------------------------------------- 4. Some related history: As of early 2011, SquashFS-LZMA was the only format we'd found that worked for the purposes of this package. Other formats exhibited prob- lems of various types: (a) In theory, the ZIP format (in conjunction with "fusezip") should have worked. In practice, "fusezip" wasn't able to mount a ZIP version of this dataset correctly. (b) In theory, the CROMFS format (in conjunction with "cromfs- driver") should have worked. In practice, "mkcromfs" report- ed errors while it was processing this dataset, and "cromfs- driver" wasn't able to mount the resulting filesystem. (c) CRAMFS (note: separate from CROMFS) didn't support large directories. (d) ZFS supported compression. However, it didn't compress dir- ectory structures, which made it unsuitable for use with this dataset. Additionally, ZFS had high memory require- ments. (e) There were other filesystems that supported compression, but the ones checked had one or both of the ZFS problems men- tioned above. So this distro used LZM (SquashFS-LZMA) format for about a year and a half. One disadvantage of the LZM format was that nested mounts didn't work; so, if an LZM file was used in a LiveDistro, it couldn't be stored inside any of the ".lzm" files that made up the LiveDistro at the time. It needed to be stored as a separate file on the boot device or on an external storage device. Another disadvantage was that LZM support was fragile; for example, it required kernel patches. Around mid-2011, the Linux kernel started to provide native support for the XZM (SquashFS-XZ) format, which was similar to the LZM format and solved both of these problems. So this distro switched from LZM to XZM in the Fall. #--------------------------------------------------------------------- 5. Warning: MNTDIR, in this file's "build" block, must meet the fol- lowing conditions: (a) The string must be equal to "/mnt/cddbdata". (b) The string must specify the absolute pathname of a folder. (c) Prior to initiation of a "build", a filesystem that has a large amount of free disk space and a large number of free inodes must be mounted on the folder. 16GB of free disk space and 4,000,000 free inodes may be needed. (d) The filesystem must be located on a fast disk drive. USB3 may be acceptable, but USB2 is inadvisable. (e) The filesystem used must be writable. Alternatively, nothing needs to be mounted on the folder, provided that the "build" system's root filesystem meets the conditions listed above. Additional warning: The initial contents of the specified folder will be deleted. The "build" system should have modern hardware, dated 2013 or later if possible. RAM should be at least 4GB. 8GB or 16GB may work better. If all of the conditions are met, the "exit 0" statement that occurs at a later point may be commented out. Otherwise, the statement in question should be uncommented. Build times may vary considerably depending on the types of filesys- tems used. #--------------------------------------------------------------------- 6. At "build" time, this package requires only a small amount of free disk space in the "build" directory used. However, if "/mnt/cddbdata/" exists and a filesystem is mounted on it, the filesystem must have at least 16GB of free disk space. The primary filesystem must have at least 16GB of free disk space other- wise. An additional 1.1GB of free disk space is needed in "/mnt/xzm/" for the purposes of storage of the XZM file created. #--------------------------------------------------------------------- [buildtimes] 00.52 hours (or 0031.33 minutes) - HP EliteBook 8560w 32GB RAM 00.86 hours (or 0051.37 minutes) - 2021 L.A. dedi box 6x2 CPU 32GB RAM 01.15 hours (or 0069.00 minutes) - ThinkPad W530 i7 4x2 32GB RAM 22.00 hours (or 1320.00 minutes) - HP Pavilion dv8 8GB RAM 24.26 hours (or 1455.73 minutes) - ThinkPad E540 i7 4x2 16GB RAM #--------------------------------------------------------------------- [settings] configure = none exepack = yes license = GNU General Public License, version 2 (June 1991) nosyslinks = yes tmpsize = 2M unpack = relaxed #--------------------------------------------------------------------- [depends] libcddb # Probably isn't required sed squashfs-tools #--------------------------------------------------------------------- [build] chown -R 0.0 . sh ./config.sh pwd #--------------------------------------------------------------------- MNTDIR=/mnt/cddbdata XZMDIR=/xzm XZMBASE=cddbdata.xzm XZMPATH=$XZMDIR/$XZMBASE killany "cddbd -s" || true umount $XZMPATH 2> /dev/null || true mkdir -p $MNTDIR touch $MNTDIR/doge rm -fr $MNTDIR/* mkdir -p $MNTDIR mkdir -p $XZMDIR rm -fr $XZMPATH make bash ./install.sh cd $PKGDIR_PROD/sbin sed -e "s|__META_LOOPFILE__|$XZMPATH|g" \ -e "s|__META_MNTDIR__|$MNTDIR|g" \ -e "s|__META_PREFIX__|$PKGDIR_PROD|g" \ -e "s|__META_PRODTREE__|$PRODTREE|g" \ < $PKGDIR_SPECIAL/cddbd-start.in \ > cddbd-start chmod 755 cddbd-start cd /laclin/lacutil sed -e "s|__META_PREFIX__|$PKGDIR_PROD|g" \ -e "s|__META_PRODTREE__|$PRODTREE|g" \ < $PKGDIR_SPECIAL/cddbd-sudo.in \ > cddbd-sudo chmod 755 cddbd-sudo #-------------------------------------------------------------------- #ifdef NOTDEF # cd /usr/local/pages/util # sed -e "s|__META_PREFIX__|$PKGDIR_PROD|g" \ # -e "s|__META_PRODTREE__|$PRODTREE|g" \ # < $PKGDIR_SPECIAL/cddbd.cgi.in \ # > cddbd.cgi # chmod 755 cddbd.cgi #endif #-------------------------------------------------------------------- # The "exit 0" statement below suppresses rebuilds of the ".xzm" file # associated with this package. # If you'd like to try to rebuild the file, comment-out the statement # in question. # If you don't have high system resources, you should uncomment the # statement instead. In this event, you should remove installed cop- # ies of the ".xzm" file unless you're sure that they're complete and # appropriate for your architecture. exit 0 #-------------------------------------------------------------------- # Directory containing FreeDB tarball BALLDIR=$PKGDIR_SPECIAL/freedb-complete # Tarball name (without path) BALLNAME=freedb-complete-20200601.tar.bz2 BALLPATH=$BALLDIR/$BALLNAME # Tarball pathname export BUILDING=yes # Needed by START # Daemon startup command START=$PRODTREE/cddbd/sbin/cddbd-start # Command that builds "fuzzy" index MAKEFUZZY="$PRODTREE/cddbd/sbin/cddbd -df" BSIZE=1048576 # "mksquashfs" block size TMPDIR=/ram # Main temporary directory (needs a # lot of free disk space) export TMPDIR # May be used by some lower-level pro- # grams #-------------------------------------------------------------------- echo ==== `date` freedb conversion procedure started echo ==== `date` "touch $TMPDIR/FREEDBSTART" touch $TMPDIR/FREEDBSTART echo ==== `date` "cd $MNTDIR" cd $MNTDIR echo ==== `date` "tar jxf $BALLPATH" tar jxf $BALLPATH # The next step is needed due to errors in the data. # rm -fr blues/4011ca13 rm -fr data/260e5d15 rm -fr data/49126216 rm -fr data/e409de11 rm -fr folk/8209af0b rm -fr misc/9507b80a rm -fr misc/950b910c rm -fr rock/51094907 rm -fr rock/a40be10d rm -fr rock/d008660f echo ==== `date` "chown -R 0.0 ." chown -R 0.0 . #ifdef NOTDEF # echo ==== `date` "settreedate ." # settreedate . #endif echo ==== `date` $START $START echo ==== `date` $MAKEFUZZY $MAKEFUZZY echo ==== `date` "chmod 644 fuzzy_index" chmod 644 fuzzy_index echo ==== `date` "rm -fr /mnt/cddbdata/temp.fuzzy" rm -fr /mnt/cddbdata/temp.fuzzy echo ==== `date` "mksquashfs . $XZMPATH -b $BSIZE" mksquashfs . $XZMPATH -b $BSIZE echo ==== `date` "chmod 644 $XZMPATH" chmod 644 $XZMPATH echo ==== `date` "cd /" cd / echo ==== `date` "rm -fr $MNTDIR/*" rm -fr $MNTDIR/* echo ==== `date` "umount $MNTDIR 2>/dev/null" umount $MNTDIR 2>/dev/null || true echo ==== `date` "ls -l $XZMPATH" ls -l $XZMPATH echo ==== `date` "touch $TMPDIR/FREEDBFINISH" touch $TMPDIR/FREEDBFINISH echo ==== `date` "filetimedelta $TMPDIR/FREEDB{START,FINISH}" filetimedelta $TMPDIR/FREEDB{START,FINISH} \ echo ==== `date` freedb conversion procedure finished #--------------------------------------------------------------------- [setup.boot] if [ -d $PRODTREE/cddbd ]; then killany "cddbd -s" MNTDIR=/mnt/cddbdata mkdir -p $MNTDIR umount $MNTDIR >& /dev/null chmod 777 $MNTDIR /laclin/lacutil/xzm-boot-setup cddbdata EXTERNAL="freedb\.freedb\.org" sed -e "s|^\(127\.0\.0\.1.*$EXTERNAL\)|# \1|" -i /etc/hosts fi #--------------------------------------------------------------------- # Original URLs. These URLs were valid at one point, but may have died # since then. If you download newer versions of tarballs [etc.], don't # delete the original versions, as you may not be able to replace # them. [urls] url_debian = Not checked yet url_home = http://www.freedb.org/ url_lfs = Not checked yet url_data = special # See "*.url" in "special/" url_tarball = n/a # Orphaned