Linux: create arbitrary directory / file hierarchy

To test the tool, I need a directory with a number of different Office files in a deep nested structure. I already have the files in the directory, but now you need to create some random nested subdirectories and decompose the files into them.

I could sit down and write a suitable program in the programming language of my choice, but I wonder if there can be a clever combination of Linux + Bash command line tools to achieve what I want.

Edit: to clarify, my input is a directory with about 200 files. The result should be a directory hierarchy containing these files, more or less evenly distributed. Directory names must be more than one letter, randomly vary in length and use different valid characters (utf-8 file system).

+6
source share
4 answers

I was not too happy with these answers, so I came up with my own. The following takes my input files and uses / dev / urandom to collect from 10 to 256 printed characters, adds a few more directory separators, creates a directory hierarchy and puts the file in it.

Using urandom creates some really strange directory names that are good for my purpose. I am sure that a true Unix guru can simplify this even further. For example, building dir can be done in a single awk command.

#!/bin/bash INDIR='files'; IFS=$'\n' for FILE in `ls $INDIR/*`; do DIR=`cat /dev/urandom | \ tr -dc '[ -~]' | \ tr 'ABCDEF\\\\' '///////' | \ head -c$((10 + $RANDOM % 256))` mkdir -p $DIR cp $FILE $DIR done 
+2
source

You can use bash brace-expansion :

 mkdir -p {a,b}/{e,f,g}/{h,i,j} 

 β”œβ”€β”€β”€a β”‚ β”œβ”€β”€β”€e β”‚ β”‚ β”œβ”€β”€β”€h β”‚ β”‚ β”œβ”€β”€β”€i β”‚ β”‚ └───j β”‚ β”œβ”€β”€β”€f β”‚ β”‚ β”œβ”€β”€β”€h β”‚ β”‚ β”œβ”€β”€β”€i β”‚ β”‚ └───j β”‚ └───g β”‚ β”œβ”€β”€β”€h β”‚ β”œβ”€β”€β”€i β”‚ └───j └───b β”œβ”€β”€β”€e β”‚ β”œβ”€β”€β”€h β”‚ β”œβ”€β”€β”€i β”‚ └───j β”œβ”€β”€β”€f β”‚ β”œβ”€β”€β”€h β”‚ β”œβ”€β”€β”€i β”‚ └───j └───g β”œβ”€β”€β”€h β”œβ”€β”€β”€i └───j 
+8
source

These are scripts that generate an arbitrary dir structure:

 #!/bin/bash # Decimal ASCII codes (see man ascii) ARR=( {48..57} {65..90} {97..122} ) # Array count arrcount=${#ARR[@]} # return a random string get_rand_dir(){ for ((i=1; i<$((RANDOM%30)); i++)) { printf \\$(printf '%03o' ${ARR[RANDOM%arrcount]}); } } dir=/tmp/ # appending random characters to make a hierarchy for ((i=0; i<$((RANDOM%100)); i++)) { dir+="$(get_rand_dir)/" } echo $dir mkdir -p "$dir" oldir=$(echo "$dir" | cut -d '/' -f1-3) while [[ $dir ]]; do dir=${dir%/*} cd $dir for ((i=0; i<$((RANDOM%100)); i++)) { mkdir &>/dev/null -p $(get_rand_dir) } done tree "$oldir" 

OUTPUT

 /tmp/x β”œβ”€β”€ egeDVPW β”œβ”€β”€ iOkr β”œβ”€β”€ l β”œβ”€β”€ o1gye8uF β”œβ”€β”€ q β”‚  β”œβ”€β”€ 4Dlrfagv β”‚  β”œβ”€β”€ 4Yxmoqf β”‚  β”œβ”€β”€ 8LkyIrXA β”‚  β”œβ”€β”€ 8m9kse8s β”‚  β”œβ”€β”€ aV β”‚  β”œβ”€β”€ in β”‚  β”‚  β”œβ”€β”€ 12zdLso68HWlPK β”‚  β”‚  β”‚  β”œβ”€β”€ C β”‚  β”‚  β”‚  β”œβ”€β”€ DOYt8wUW β”‚  β”‚  β”‚  β”œβ”€β”€ FXP β”‚  β”‚  β”‚  β”œβ”€β”€ hFLem8 β”‚  β”‚  β”‚  β”œβ”€β”€ hhHIv β”‚  β”‚  β”‚  β”œβ”€β”€ iD87kxs54x04 β”‚  β”‚  β”‚  β”œβ”€β”€ oFM β”‚  β”‚  β”‚  β”œβ”€β”€ OjFT 

Now you can create an array of dirs:

 shopt -s globstar # require bash4 dirs=( /tmp/x/** ) printf '%s\n' ${dirs[@]} 

and randomly fill disks with files. You have enough examples for this. I did a great job.

+5
source

Thanks to everyone who posted here; it turns out it’s actually trivial to avoid special character file names, so I built my own script based on these here; Here's how it behaves with special character names:

 $ ~/rndtree.sh ./rndpath 0 3 1 Warning: will create random tree at: ./rndpath Proceed (y/n)? y Removing old outdir ./rndpath mkdir -p ./rndpath/";"/{")?DxVBBJ{w2","L,|+","^VC)Vn.6!"}/"D+,IFJ( LN" > > > > > > > > > > > ./rndpath └── [ 4096] ; β”œβ”€β”€ [ 4096] )?DxVBBJ{w2 β”‚  β”œβ”€β”€ [ 4096] D+,IFJ( LN β”‚  β”‚  └── [ 929] r2.bin β”‚  β”œβ”€β”€ [ 8557] %3fsaG# Rl;ffXf.bin β”‚  └── [ 19945] Dzk .bin β”œβ”€β”€ [ 4096] L,|+ β”‚  β”œβ”€β”€ [ 4096] D+,IFJ( LN β”‚  β”‚  β”œβ”€β”€ [ 2325] 6Qg#pe5j'&ji49oqrO.bin β”‚  β”‚  β”œβ”€β”€ [ 16345] #?.bin β”‚  β”‚  └── [ 2057] Uz-0XtLVWz#}0lI.bin β”‚  β”œβ”€β”€ [ 2543] bbtA-^s22vdTu.bin β”‚  └── [ 10848] K46+kh7L9.bin β”œβ”€β”€ [ 4096] ^VC)Vn.6! β”‚  β”œβ”€β”€ [ 4096] D+,IFJ( LN β”‚  β”œβ”€β”€ [ 10502] 8yY,MqZ ^5+_SA^.r4{.bin β”‚  └── [ 17628] ipO"|69.bin └── [ 12376] a2Y% }G1.qDir.bin 7 directories, 11 files total bytes: 136823 ./rndpath 

and here with a safe subset of ASCII:

 $ ~/rndtree.sh ./rndpath 1 3 1 Warning: will create random tree at: ./rndpath Proceed (y/n)? y Removing old outdir ./rndpath mkdir -p ./rndpath/"48SLS"/{"nyG","jIC6vj"}/{"PSLd5tMn","VR"} > > > > > > > ./rndpath β”œβ”€β”€ [ 4096] 48SLS β”‚  β”œβ”€β”€ [ 4096] jIC6vj β”‚  β”‚  β”œβ”€β”€ [ 4096] PSLd5tMn β”‚  β”‚  β”œβ”€β”€ [ 4096] VR β”‚  β”‚  β”‚  β”œβ”€β”€ [ 922] lg.bin β”‚  β”‚  β”‚  └── [ 9600] VVYG.bin β”‚  β”‚  β”œβ”€β”€ [ 10883] B7nt06p.bin β”‚  β”‚  └── [ 19339] g5uT i.bin β”‚  β”œβ”€β”€ [ 4096] nyG β”‚  β”‚  β”œβ”€β”€ [ 4096] PSLd5tMn β”‚  β”‚  └── [ 4096] VR β”‚  β”‚  └── [ 6128] 1tfLR.bin β”‚  └── [ 5448] Jda.bin └── [ 18196] KwEXu2O2H9s.bin 

Spaces should be handled in both cases - however, note that the subdirectory names are repeated (while the file names do not match).

script rndtree.sh :

 #!/usr/bin/env bash # http://stackoverflow.com/questions/13400312/linux-create-random-directory-file-hierarchy # Decimal ASCII codes (see man ascii); added space AARR=( 32 {48..57} {65..90} {97..122} ) # Array count aarrcount=${#AARR[@]} if [ "$1" == "" ] ; then OUTDIR="./rndpath" ; else OUTDIR="$1" ; fi if [ "$2" != "" ] ; then ASCIIONLY="$2" ; else ASCIIONLY=1 ; fi if [ "$3" != "" ] ; then DIRDEPTH="$3" ; else DIRDEPTH=3 ; fi if [ "$4" != "" ] ; then MAXFIRSTLEVELDIRS="$4" ; else MAXFIRSTLEVELDIRS=2 ; fi if [ "$5" != "" ] ; then MAXDIRCHILDREN="$5" ; else MAXDIRCHILDREN=4 ; fi if [ "$6" != "" ] ; then MAXDIRNAMELEN="$6" ; else MAXDIRNAMELEN=12 ; fi if [ "$7" != "" ] ; then MAXFILECHILDREN="$7" ; else MAXFILECHILDREN=4 ; fi if [ "$8" != "" ] ; then MAXFILENAMELEN="$8" ; else MAXFILENAMELEN=20 ; fi if [ "$9" != "" ] ; then MAXFILESIZE="$9" ; else MAXFILESIZE=20000 ; fi MINDIRNAMELEN=1 MINFILENAMELEN=1 MINDIRCHILDREN=1 MINFILECHILDREN=0 MINFILESIZE=1 FILEEXT=".bin" VERBOSE=0 #1 get_rand_dirname() { if [ "$ASCIIONLY" == "1" ]; then for ((i=0; i<$((MINDIRNAMELEN+RANDOM%MAXDIRNAMELEN)); i++)) { printf \\$(printf '%03o' ${AARR[RANDOM%aarrcount]}); } else cat /dev/urandom | tr -dc '[ -~]' | tr -d '[$></~:`\\]' | head -c$((MINDIRNAMELEN + RANDOM % MAXDIRNAMELEN)) | sed 's/\(["]\)/\\\1/g' fi #echo -e " " # debug last dirname space } get_rand_filename() { if [ "$ASCIIONLY" == "1" ]; then for ((i=0; i<$((MINFILENAMELEN+RANDOM%MAXFILENAMELEN)); i++)) { printf \\$(printf '%03o' ${AARR[RANDOM%aarrcount]}); } else # no need to escape double quotes for filename cat /dev/urandom | tr -dc '[ -~]' | tr -d '[$></~:`\\]' | head -c$((MINFILENAMELEN + RANDOM % MAXFILENAMELEN)) #| sed 's/\(["]\)/\\\1/g' fi printf "%s" $FILEEXT } echo "Warning: will create random tree at: $OUTDIR" [ "$VERBOSE" == "1" ] && echo " MAXFIRSTLEVELDIRS $MAXFIRSTLEVELDIRS ASCIIONLY $ASCIIONLY DIRDEPTH $DIRDEPTH MAXDIRCHILDREN $MAXDIRCHILDREN MAXDIRNAMELEN $MAXDIRNAMELEN MAXFILECHILDREN $MAXFILECHILDREN MAXFILENAMELEN $MAXFILENAMELEN MAXFILESIZE $MAXFILESIZE" read -p "Proceed (y/n)? " READANS if [ "$READANS" != "y" ]; then exit fi if [ -d "$OUTDIR" ]; then echo "Removing old outdir $OUTDIR" rm -rf "$OUTDIR" fi mkdir "$OUTDIR" if [ $MAXFIRSTLEVELDIRS -gt 0 ]; then NUMFIRSTLEVELDIRS=$((1+RANDOM%MAXFIRSTLEVELDIRS)) else NUMFIRSTLEVELDIRS=0 fi # create directories for (( ifl=0;ifl<$((NUMFIRSTLEVELDIRS));ifl++ )) { FLDIR="$(get_rand_dirname)" FLCHILDREN=""; for (( ird=0;ird<$((DIRDEPTH-1));ird++ )) { DIRCHILDREN=""; MOREDC=0; for ((idc=0; idc<$((MINDIRCHILDREN+RANDOM%MAXDIRCHILDREN)); idc++)) { CDIR="$(get_rand_dirname)" ; # make sure comma is last, so brace expansion works even for 1 element? that can mess with expansion math, though if [ "$DIRCHILDREN" == "" ]; then DIRCHILDREN="\"$CDIR\"" ; else DIRCHILDREN="$DIRCHILDREN,\"$CDIR\"" ; MOREDC=1 ; fi } if [ "$MOREDC" == "1" ] ; then if [ "$FLCHILDREN" == "" ]; then FLCHILDREN="{$DIRCHILDREN}" ; else FLCHILDREN="$FLCHILDREN/{$DIRCHILDREN}" ; fi else if [ "$FLCHILDREN" == "" ]; then FLCHILDREN="$DIRCHILDREN" ; else FLCHILDREN="$FLCHILDREN/$DIRCHILDREN" ; fi fi } DIRCMD="mkdir -p $OUTDIR/\"$FLDIR\"/$FLCHILDREN" eval "$DIRCMD" echo "$DIRCMD" } # now loop through all directories, create random files inside # note printf '%q' escapes to preserve spaces; also here # escape, and don't wrap path parts in double quotes (eg | sed 's_/_"/"_g'); # note then we STILL have to eval to use it! # but now ls "$D" works, so noneed for QD # unfortunately backslashes can make '%q' barf - prevent them find "$OUTDIR" -type d | while IFS= read D ; do QD="$(printf '%q' "$(echo "$D")" )" ; [ "$VERBOSE" == "1" ] && echo "$D"; #echo "$QD"; ls -la "$D"; #eval "ls -la $QD"; for ((ifc=0; ifc<$((MINFILECHILDREN+RANDOM%MAXFILECHILDREN)); ifc++)) { CFILE="$(get_rand_filename)" ; echo -n '> ' [ "$VERBOSE" == "1" ] && echo "$D"/"$CFILE" cat /dev/urandom \ | head -c$((MINFILESIZE + RANDOM % MAXFILESIZE)) \ > "$D"/"$CFILE" } done echo tree -a --dirsfirst -s "$OUTDIR" echo "total bytes: $(du -bs $(echo "$OUTDIR"))" 
+3
source

All Articles