JeanBonnot Posted March 31, 2022 Share Posted March 31, 2022 Is there a backup and restore script on armbian using rsync and NFS for backup entire system to NAS like there is on dietpi (dietpi-backup) which is excellent? 0 Quote Link to comment Share on other sites More sharing options...
TRS-80 Posted March 31, 2022 Share Posted March 31, 2022 Not that I am aware of. Armbian is focused on much lower level things, like kernels, etc. Have you studied the script? Maybe you could adapt it (or even use it directly)? 0 Quote Link to comment Share on other sites More sharing options...
lanefu Posted April 1, 2022 Share Posted April 1, 2022 10 hours ago, JeanBonnot said: Is there a backup and restore script on armbian using rsync and NFS for backup entire system to NAS like there is on dietpi (dietpi-backup) which is excellent? That's beyond the feature-scope of Armbian. And yes a great feature on diet-pi, which makes a very nice appliance experience for the user..... part of it's appliciance-ification approach to config management is also what lets it back up so nicely. 0 Quote Link to comment Share on other sites More sharing options...
JeanBonnot Posted April 1, 2022 Author Share Posted April 1, 2022 (edited) 18 hours ago, TRS-80 said: Not that I am aware of. Armbian is focused on much lower level things, like kernels, etc. Have you studied the script? Maybe you could adapt it (or even use it directly)? I'll try, but I'm a poor bash script developer 16 hours ago, lanefu said: That's beyond the feature-scope of Armbian. And yes a great feature on diet-pi, which makes a very nice appliance experience for the user..... part of it's appliciance-ification approach to config management is also what lets it back up so nicely. As u say 😉 This is the script dietpi-backup : Spoiler #!/bin/bash { #//////////////////////////////////// # DietPi Backup # #//////////////////////////////////// # Created by Daniel Knight / daniel.knight@dietpi.com / dietpi.com # #//////////////////////////////////// # # Info: # - Location: /boot/dietpi/dietpi-backup # - Allows for a complete system back and restore of the linux filesystem (/) # # Usage: # - dietpi-backup -1 = Restore # - dietpi-backup = Menu # - dietpi-backup 1 = Backup # # $2 = optional directory location to use with backup/restore input: # - dietpi-backup -1 /path/to/target = Restore # - dietpi-backup 1 /path/to/target = Backup #//////////////////////////////////// # Import DietPi-Globals -------------------------------------------------------------- . /boot/dietpi/func/dietpi-globals readonly G_PROGRAM_NAME='DietPi-Backup' G_CHECK_ROOT_USER G_CHECK_ROOTFS_RW G_INIT # Import DietPi-Globals -------------------------------------------------------------- # Grab input [[ $1 =~ ^-?1$ ]] && INPUT=$1 || INPUT=0 #///////////////////////////////////////////////////////////////////////////////////// # Backup System #///////////////////////////////////////////////////////////////////////////////////// # Backup file paths FP_SOURCE='/' FP_TARGET='/mnt/dietpi-backup' # Stats and transfer logs, stored to: $FP_TARGET/ readonly FP_LOG='dietpi-backup.log' readonly FP_STATS='.dietpi-backup_stats' # Include/exclude file readonly FP_FILTER='.dietpi-backup_filter_inc_exc' readonly FP_FILTER_CUSTOM='/boot/dietpi/.dietpi-backup_inc_exc' # rsync options # - Backup: Delete files in target which are not present in source, or excluded readonly aRSYNC_RUN_OPTIONS_BACKUP=('-aH' '--info=name0' '--info=progress2' '--delete-excluded' "--exclude-from=$FP_FILTER") # - Restore: Delete files in target which are not present in source, but after the transfer has finished, and leave excluded files untouched readonly aRSYNC_RUN_OPTIONS_RESTORE=('-aH' '--info=name0' '--info=progress2' '--delete-after' "--exclude-from=$FP_FILTER") # Date format for logs Print_Date(){ date '+%Y-%m-%d_%T'; } # rsync already running error: $1=Backup/Restore Error_Rsync_Already_Running(){ G_DIETPI-NOTIFY 1 'Another rsync process is already running.' echo -e "$1 failed: $(Print_Date). rsync is already running." >> "$FP_TARGET/$FP_STATS" G_WHIP_MSG "$1 Error:\n\nA $1 could not be started as rsync is already running." /boot/dietpi/dietpi-services start } Check_Supported_Directory_Location(){ # Obtain filesystem type. Create directory temporarily for this if missing. if [[ ! -d $FP_TARGET ]] then G_EXEC mkdir -p "$FP_TARGET" local fs_type=$(df -T "$FP_TARGET" | mawk 'NR==2 {print $2}') G_EXEC rmdir "$FP_TARGET" else local fs_type=$(df -T "$FP_TARGET" | mawk 'NR==2 {print $2}') fi # Check for supported filesystem type if [[ $fs_type =~ ^(ext[2-4]|(f2|btr|x|z)fs)$ ]] then return 0 elif [[ $fs_type =~ ^nfs4?$ ]] then G_CHECK_FS_PERMISSION_SUPPORT "$FP_TARGET" && return 0 G_DIETPI-NOTIFY 1 "NFS mount $FP_TARGET does not support UNIX permissions" G_WHIP_MSG "Unsupported NFS share:\n\n$FP_TARGET is an NFS share which does not support UNIX permissions.\n\nMake sure that the NFS share at the server is located on a filesystem with native symlink and UNIX permissions support." else G_DIETPI-NOTIFY 1 "Filesystem type $fs_type not supported in $FP_TARGET" G_WHIP_MSG "Filesystem type not supported:\n\n$FP_TARGET has a filesystem type of $fs_type, which is not supported.\n\nThe filesystem type must be ext2/3/4, F2FS, Btrfs, XFS, ZFS or a proper NFS mount with native symlink and UNIX permissions support." fi return 1 } Create_Filter_Include_Exclude(){ cat << _EOF_ > $FP_FILTER # Backup data, log and config - $FP_TARGET/ - $FP_SETTINGS # RAM dirs - /dev/ - /proc/ - /run/ - /sys/ - /tmp/ # Swap files - /var/swap - .swap* # Fake RTC timestamp - /etc/fake-hwclock.data # Unlinked inodes - /lost+found/ # APT cache - /var/cache/apt/* _EOF_ # Add users filter list [[ -f $FP_FILTER_CUSTOM ]] && cat $FP_FILTER_CUSTOM >> $FP_FILTER } Run_Backup(){ G_DIETPI-NOTIFY 3 "$G_PROGRAM_NAME" 'Backup' # Check valid FS Check_Supported_Directory_Location || return 1 # Rotate backups with more than 1 shall be kept if (( $AMOUNT > 1 )) then G_DIETPI-NOTIFY 2 'Rotating backup archive' if [[ -d $FP_TARGET/data_$AMOUNT ]] then [[ -d $FP_TARGET/data_tmp ]] && G_EXEC rm -R "$FP_TARGET/data_tmp" # Failsafe G_EXEC mv "$FP_TARGET/data_$AMOUNT" "$FP_TARGET/data_tmp" fi if [[ -d $FP_TARGET/data ]] then for ((i=$AMOUNT-1;i>1;i--)) do [[ -d $FP_TARGET/data_$i ]] && G_EXEC mv "$FP_TARGET/data_$i" "$FP_TARGET/data_$((i+1))" done G_EXEC mv "$FP_TARGET/data" "$FP_TARGET/data_2" fi [[ -d $FP_TARGET/data_tmp ]] && G_EXEC mv "$FP_TARGET/data_tmp" "$FP_TARGET/data" fi # Remove backups above selected amount local number for i in "$FP_TARGET/data_"* do number=${i#"$FP_TARGET/data_"} disable_error=1 G_CHECK_VALIDINT "$number" $(($AMOUNT+1)) || continue G_EXEC_DESC="Removing backup number $number which is above the selected amount $AMOUNT" G_EXEC rm -R "$i" done # Generate target directory if missing [[ -d $FP_TARGET/data ]] || G_EXEC mkdir -p "$FP_TARGET/data" /boot/dietpi/dietpi-services stop # Check if rsync is already running, while the daemon should have been stopped above pgrep 'rsync' &> /dev/null && { Error_Rsync_Already_Running 'Backup'; return 1; } # Install required rsync if missing G_AG_CHECK_INSTALL_PREREQ rsync # Generate Exclude/Include lists Create_Filter_Include_Exclude G_DIETPI-NOTIFY 2 'Checking for sufficient disk space via rsync dry-run, please wait...' local old_backup_size=$(du -sB1 "$FP_TARGET/data" | mawk '{print $1}') # Actual disk usage in bytes # - Dry run to obtain transferred data size rsync --dry-run --stats "${aRSYNC_RUN_OPTIONS_BACKUP[@]}" "$FP_SOURCE" "$FP_TARGET/data/" > .dietpi-backup_result local new_backup_size=$(grep -m1 '^Total file size:' .dietpi-backup_result | sed 's/[^0-9]*//g') # Apparent data size without block size related overhead local total_file_count=$(mawk '/^Number of files:/{print $6;exit}' .dietpi-backup_result | sed 's/[^0-9]*//g') local total_folder_count=$(mawk '/^Number of files:/{print $8;exit}' .dietpi-backup_result | sed 's/[^0-9]*//g') rm .dietpi-backup_result local target_fs_blocksize=$(stat -fc '%s' "$FP_TARGET/data") new_backup_size=$(( $new_backup_size + ( $total_file_count + $total_folder_count ) * $target_fs_blocksize )) # Add one block size for each file + dir as worst case result local end_result=$(( ( $new_backup_size - $old_backup_size ) / 1024**2 + 1 )) # bytes => MiB rounded up # - Perform check if ! G_CHECK_FREESPACE "$FP_TARGET/data" $end_result; then G_WHIP_BUTTON_OK_TEXT='Ignore' G_WHIP_BUTTON_CANCEL_TEXT='Exit' if ! G_WHIP_YESNO 'The backup target location appears to have insufficient free space to successfully finish the backup. However, this check is a rough estimation in reasonable time, thus it could be marginally incorrect. \nWould you like to override this warning and continue with the backup?'; then echo -e "Backup cancelled due to insufficient free space : $(Print_Date)" >> "$FP_TARGET/$FP_STATS" /boot/dietpi/dietpi-services start return 1 fi fi G_DIETPI-NOTIFY 2 "Backup to $FP_TARGET in progress, please wait..." # Init log file echo -e "Backup log from $(Print_Date)\n" > "$FP_TARGET/$FP_LOG" rsync "${aRSYNC_RUN_OPTIONS_BACKUP[@]}" -v --log-file="$FP_TARGET/$FP_LOG" "$FP_SOURCE" "$FP_TARGET/data/" EXIT_CODE=$? # touch target directory to show the correct last update timestamp when restoring one of multiple backups from the archive. This needs to be done after the backup since it applies the root / timestamps. G_EXEC touch "$FP_TARGET/data" /boot/dietpi/dietpi-services start G_DIETPI-NOTIFY -1 $EXIT_CODE "$G_PROGRAM_NAME: Backup" if (( $EXIT_CODE == 0 )); then echo -e "Backup completed : $(Print_Date)" >> "$FP_TARGET/$FP_STATS" G_WHIP_MSG "Backup completed:\n - $FP_TARGET" else G_WHIP_MSG "Backup failed:\n - $FP_TARGET\n\nYou will see the log file on the next screen. Please check it for information and/or errors." fi log=1 G_WHIP_VIEWFILE "$FP_TARGET/$FP_LOG" } # When restoring a backup, assure that either the UUIDs stored in the backup fstab/boot config matches the current system drive, or that we know all relevant files to adjust afterwards. Check_UUIDs() { UPDATE_UUIDs=0 UPDATE_GRUB=0 UPDATE_RPI=0 UPDATE_ARMBIAN=0 UPDATE_ODROID=0 UPDATE_UBOOT=0 UUID_ROOT=$(findmnt -Ufnro UUID -M /) PARTUUID_ROOT=$(findmnt -Ufnro PARTUUID -M /) # If the current rootfs' UUID or PARTUUID can be found in the the backups fstab, it can be assumed that it was created from the same drives. grep -q "^UUID=${UUID_ROOT}[[:blank:]]" "$FP_TARGET/$DATA/etc/fstab" || grep -q "^PARTUUID=${PARTUUID_ROOT}[[:blank:]]" "$FP_TARGET/$DATA/etc/fstab" && return 0 UPDATE_UUIDs=1 # Else check if we know how to adjust the boot config after the backup has been restored. # - x86_64 if (( $G_HW_ARCH == 10 )) && command -v update-grub > /dev/null then UPDATE_GRUB=1 # - RPi elif (( $G_HW_MODEL < 10 )) && [[ -f $FP_TARGET/$DATA/boot/cmdline.txt ]] then UPDATE_RPI=1 # - Armbian elif [[ -f $FP_TARGET/$DATA/boot/armbianEnv.txt ]] then UPDATE_ARMBIAN=1 # - Odroids / classic U-Boot elif [[ -f $FP_TARGET/$DATA/boot/boot.ini ]] then UPDATE_ODROID=1 # - Modern U-Boot elif [[ -f $FP_TARGET/$DATA/boot/boot.cmd ]] && command -v mkimage > /dev/null then UPDATE_UBOOT=1 # - Else we cannot assure that the restored image will boot. else # Let user decide, but default to "no" G_WHIP_YESNO '[WARNING] UUIDs of the backup and the current system differ \nThe file systems unique identifiers, usually used to mount the drives at boot, seem to differ between the backup and the current system. \nThis usually indicates that you try to restore an old backup onto a newly flashed DietPi system. \nMoreover are we not able to find the boot configuration, where those UUIDs would need to be adjusted, to assure that the system will boot. \nWe hence do not recommend to restore this backup on this system. If you continue, you will need to assure yourself that fstab and boot configurations match the UUIDs, else the system may not boot. \nDo you want to restore this backup?' && return 0 || return 1 fi G_WHIP_DEFAULT_ITEM='ok' G_WHIP_YESNO '[WARNING] UUIDs of the backup and the current system differ \nThe file systems unique identifiers, usually used to mount the drives at boot, seem to differ between the backup and the current system. \nThis usually indicates that you try to restore an old backup onto a newly flashed DietPi system. \nBut we were able to find the boot configuration, where those UUIDs would need to be adjusted, to assure that the system will boot. \nIt should be hence safe to restore this backup, but if the UUIDs were used elsewhere, you might need to adjust it manually. \nDo you want to restore this backup?' && return 0 || return 1 } Update_UUIDs() { # fstab while read -r mountpoint do [[ $mountpoint ]] || continue local uuid=$(findmnt -Ufnro UUID -M "$mountpoint") [[ $uuid ]] && G_EXEC sed -i "\|[[:blank:]]${mountpoint}[[:blank:]]|s|^[[:blank:]]*UUID=[^[:blank:]]*|UUID=$uuid|" /etc/fstab local partuuid=$(findmnt -Ufnro PARTUUID -M "$mountpoint") [[ $partuuid ]] && G_EXEC sed -i "\|[[:blank:]]${mountpoint}[[:blank:]]|s|^[[:blank:]]*PARTUUID=[^[:blank:]]*|PARTUUID=$partuuid|" /etc/fstab done < <(lsblk -no MOUNTPOINT "$(lsblk -npo PKNAME "$G_ROOTFS_DEV")") # boot configs # - x86_64 if (( $UPDATE_GRUB == 1 )) then G_EXEC update-grub command -v update-tirfs > /dev/null && G_EXEC update-tirfs && return command -v dracut > /dev/null && G_EXEC dracut --force && return command -v update-initramfs > /dev/null && G_EXEC update-initramfs -u # - RPi elif (( $UPDATE_RPI == 1 )) then G_EXEC sed -Ei "s/(^|[[:blank:]])root=[^[:blank:]]*/\1root=PARTUUID=$PARTUUID_ROOT/" /boot/cmdline.txt # - Armbian elif (( $UPDATE_ARMBIAN == 1 )) then grep -q '^[[:blank:]]*rootdev=UUID=' /boot/armbianEnv.txt && G_CONFIG_INJECT 'rootdev=UUID=' "rootdev=UUID=$UUID_ROOT" /boot/armbianEnv.txt && return grep -q '^[[:blank:]]*rootdev=PARTUUID=' /boot/armbianEnv.txt && G_CONFIG_INJECT 'rootdev=PARTUUID=' "rootdev=PARTUUID=$PARTUUID_ROOT" /boot/armbianEnv.txt # - Odroids / classic U-Boot elif (( $UPDATE_ODROID == 1 )) then G_EXEC sed -Ei "s/(\"|root=)UUID=[^[:blank:]\"]*/\1UUID=$UUID_ROOT/" /boot/boot.ini G_EXEC sed -Ei "s/(\"|root=)PARTUUID=[^[:blank:]\"]*/\1PARTUUID=$PARTUUID_ROOT/" /boot/boot.ini # - Modern U-Boot elif (( $UPDATE_UBOOT == 1 )) then G_EXEC sed -Ei "s/(\"|root=)UUID=[^[:blank:]\"]*/\1UUID=$UUID_ROOT/" /boot/boot.cmd G_EXEC sed -Ei "s/(\"|root=)PARTUUID=[^[:blank:]\"]*/\1PARTUUID=$PARTUUID_ROOT/" /boot/boot.cmd G_EXEC mkimage -C none -A arm -T script -d /boot/boot.cmd /boot/boot.scr fi } # $1: Optional restore point from the archive Run_Restore(){ local DATA='data' (( ${1:-1} > 1 )) && DATA="data_$1" G_DIETPI-NOTIFY 3 "$G_PROGRAM_NAME" 'Restore' # Check valid FS Check_Supported_Directory_Location || return 1 # Error: Backup not found [[ -f $FP_TARGET/$FP_STATS ]] || { G_WHIP_MSG "Restore failed:\n\n$FP_TARGET/$FP_STATS does not exist\n\nHave you created a backup?"; return 1; } # Check for matching filesystem UUIDs of backup and running system Check_UUIDs || return 1 /boot/dietpi/dietpi-services stop # Check if rsync is already running, while the daemon should have been stopped above pgrep 'rsync' &> /dev/null && { Error_Rsync_Already_Running 'Restore'; return 1; } # Install required rsync if missing G_AG_CHECK_INSTALL_PREREQ rsync # Generate Exclude/Include lists Create_Filter_Include_Exclude G_DIETPI-NOTIFY 2 "Restore from $FP_TARGET in progress, please wait..." # Init log file echo -e "Restore log from $(Print_Date)\n" > "$FP_TARGET/$FP_LOG" rsync "${aRSYNC_RUN_OPTIONS_RESTORE[@]}" -v --log-file="$FP_TARGET/$FP_LOG" "$FP_TARGET/$DATA/" "$FP_SOURCE" EXIT_CODE=$? hash -r # Clear PATH cache (( $UPDATE_UUIDs )) && Update_UUIDs /boot/dietpi/dietpi-services start G_DIETPI-NOTIFY -1 $EXIT_CODE "$G_PROGRAM_NAME: Restore" if (( $EXIT_CODE == 0 )); then echo -e "Restore completed : $(Print_Date)" >> "$FP_TARGET/$FP_STATS" G_WHIP_MSG "Restore completed:\n - $FP_TARGET\n\nNB: A Reboot is highly recommended." else G_WHIP_MSG "Restore failed:\n - $FP_TARGET\n\nYou will see the log file on the next screen. Please check it for information and/or errors." fi log=1 G_WHIP_VIEWFILE "$FP_TARGET/$FP_LOG" } #///////////////////////////////////////////////////////////////////////////////////// # Settings #///////////////////////////////////////////////////////////////////////////////////// readonly FP_SETTINGS='/boot/dietpi/.dietpi-backup_settings' DAILY_BACKUP=0 AMOUNT=1 Write_Settings_File() { cat << _EOF_ > $FP_SETTINGS FP_TARGET=$FP_TARGET DAILY_BACKUP=$DAILY_BACKUP AMOUNT=$AMOUNT _EOF_ } Read_Settings_File(){ [[ -f $FP_SETTINGS ]] && . $FP_SETTINGS; } #///////////////////////////////////////////////////////////////////////////////////// # Menus #///////////////////////////////////////////////////////////////////////////////////// MENU_LASTITEM='Help' # Select "Help" by default TARGETMENUID=0 # Main menu EXIT_CODE=-1 # Relevant for automated calls "dietpi-backup -1" and "dietpi-backup 1" e.g. via G_PROMPT_BACKUP # TARGETMENUID=0 Menu_Main(){ local backup_last_completed='Backup not found. Please create one.' local daily_backup_text='Off' (( $DAILY_BACKUP )) && daily_backup_text='On' G_WHIP_MENU_ARRAY=( '' '●─ Info ' 'Help' ": What does $G_PROGRAM_NAME do?" ) [[ -f $FP_TARGET/$FP_LOG ]] && G_WHIP_MENU_ARRAY+=('Last log' ': Review last backup/restore log for current location') G_WHIP_MENU_ARRAY+=( '' '●─ Options ' 'Location' ': Change where your backup will be saved and restored from.' 'Filter' ': Modify include/exclude filter for backups.' 'Daily Backup' ": [$daily_backup_text] Daily backups via cron job" 'Amount' ": [$AMOUNT] The amount of backups to keep" ) if [[ -f $FP_TARGET/$FP_STATS ]] then G_WHIP_MENU_ARRAY+=('Delete' ": Remove backup ($FP_TARGET)") backup_last_completed=$(grep 'ompleted' "$FP_TARGET/$FP_STATS" | tail -1) fi G_WHIP_MENU_ARRAY+=( '' '●─ Run ' 'Backup' ': Create (or update) a backup of this device.' 'Restore' ': Restore this device from a previous backup.' ) G_WHIP_DEFAULT_ITEM=$MENU_LASTITEM G_WHIP_BUTTON_CANCEL_TEXT='Exit' if G_WHIP_MENU "Current backup and restore location:\n - $FP_TARGET\n - $backup_last_completed"; then MENU_LASTITEM=$G_WHIP_RETURNED_VALUE case "$G_WHIP_RETURNED_VALUE" in 'Help') G_WHIP_MSG 'DietPi-Backup is a program that allows you to Backup and Restore your DietPi system. \nIf you have broken your system, or want to reset your system to an earlier date, this can all be done with DietPi-Backup. \nSimply choose a location where you want to save and restore your backups from, then, select Backup or Restore. \nEnable a daily system backup to run it once a day via daily cron job. The execution time can be changed in "dietpi-cron". Note that this temporarily stops server services. Also we recommend to configure and test the backup with a manual call before enabling this feature. \nMore information: https://dietpi.com/docs/dietpi_tools/#dietpi-backup-backuprestore';; 'Last log') log=1 G_WHIP_VIEWFILE "$FP_TARGET/$FP_LOG";; 'Location') TARGETMENUID=1;; 'Filter') nano $FP_FILTER_CUSTOM;; 'Daily Backup') DAILY_BACKUP=$(( ! $DAILY_BACKUP ));; 'Amount') G_WHIP_DEFAULT_ITEM=$AMOUNT G_WHIP_INPUTBOX 'Please enter the amount of backups to keep.\n - Needs to be 1 or greater.' && G_CHECK_VALIDINT "$G_WHIP_RETURNED_VALUE" 1 && AMOUNT=$G_WHIP_RETURNED_VALUE;; 'Delete') G_WHIP_YESNO "Do you wish to DELETE the following backup?\n - $FP_TARGET" && G_EXEC_NOEXIT=1 G_EXEC rm -R "$FP_TARGET";; 'Backup') G_WHIP_YESNO "The system will be backed up to:\n - $FP_TARGET\n\nDo you wish to continue and start the backup?" && Run_Backup;; 'Restore') if (( $AMOUNT > 1 )) then G_WHIP_MENU_ARRAY=('1' "$(stat -c '%y' "$FP_TARGET/data")") for ((i=2;i<=$AMOUNT;i++)) do [[ -d $FP_TARGET/data_$i ]] && G_WHIP_MENU_ARRAY+=("$i" "$(stat -c '%y' "$FP_TARGET/data_$i")") done if (( ${#G_WHIP_MENU_ARRAY[@]} > 2 )) then G_WHIP_MENU 'Please select the backup from the archive you wish to restore:' && Run_Restore "$G_WHIP_RETURNED_VALUE" return fi fi G_WHIP_YESNO "The system will be restored from:\n - $FP_TARGET\n\nDo you wish to continue and start the restore?" && Run_Restore ;; esac else Menu_Exit fi } Menu_Exit(){ G_WHIP_SIZE_X_MAX=50 G_WHIP_YESNO "Exit $G_PROGRAM_NAME?" && TARGETMENUID=-1 EXIT_CODE=0; } # TARGETMENUID=1 Menu_Set_Directory(){ G_WHIP_MENU_ARRAY=( 'Search' ': Find previous backups' 'List' ': Select from a list of available mounts/drives' 'Manual' ': Manually type a directory to use' ) if G_WHIP_MENU "Please select the location where the backup will be saved, and restored from.\n\nYour current location:\n$FP_TARGET"; then local current_directory=$FP_TARGET case "$G_WHIP_RETURNED_VALUE" in 'Search') G_DIETPI-NOTIFY 2 'Searching for previous backups, please wait...' local alist=() mapfile -t alist < <(find / -type f -name "$FP_STATS") # Do we have any results? if [[ ${alist[0]} ]]; then # Create List for Whiptail G_WHIP_MENU_ARRAY=() for i in "${alist[@]}" do local last_backup_date=$(sed -n '/ompleted/s/^.*: //p' "$i" | tail -1) # Date of last backup for this backup local backup_directory=${i%"/$FP_STATS"} # Backup directory (minus the backup file), that we can use for target backup directory. G_WHIP_MENU_ARRAY+=("$backup_directory" ": $last_backup_date") done G_WHIP_MENU 'Please select a previous backup to use:' || return 0 FP_TARGET=$G_WHIP_RETURNED_VALUE else G_WHIP_MSG 'No previous backups were found.' return 0 fi ;; 'Manual') G_WHIP_DEFAULT_ITEM=$FP_TARGET G_WHIP_INPUTBOX 'Please enter the absolute path to the backup directory.\nE.g.: /mnt/dietpi-backup\n - Must be a filesystem which supports symlinks and UNIX permissions, like ext4, F2FS, Btrfs, XFS, ZFS or a proper NFS mount' || return 0 FP_TARGET=$G_WHIP_RETURNED_VALUE ;; 'List') /boot/dietpi/dietpi-drive_manager 1 || return 0 FP_TARGET=$(</tmp/dietpi-drive_manager_selmnt) rm /tmp/dietpi-drive_manager_selmnt [[ $FP_TARGET == '/' ]] && FP_TARGET='/mnt' FP_TARGET+='/dietpi-backup' ;; esac # If not supported, reset directory target to previous Check_Supported_Directory_Location || FP_TARGET=$current_directory else TARGETMENUID=0 # Return to main menu fi } #///////////////////////////////////////////////////////////////////////////////////// # Main Loop #///////////////////////////////////////////////////////////////////////////////////// # Read settings Read_Settings_File # $2 Optional directory input [[ $2 ]] && FP_TARGET=$2 # Create default filter file, if not yet present [[ -f $FP_FILTER_CUSTOM ]] || cat << '_EOF_' > $FP_FILTER_CUSTOM # DietPi-Backup include/exclude filter # Prefix "-" exclude items, "+" include items which would match a wildcard exclude rule. # Suffix "/" match directories only, no files or symlinks. # Using wildcard "*" matches any item name or part of it. # Since the list is processed from top to bottom and the first match defines the result, # includes need to be defined before their wildcard exclude rule # and in case excludes before their wildcard include rule. # Symlinks are handled as such and never processed recursively. # Excluded directories are not processed recursively, so contained items cannot be included. # Hence, to include items within an excluded directory: # - Do not exclude the directory itself, but contained items via wildcard. # - Define includes first, to override the wildcard exclude rule. # - See the below default rules, how we exclude all items below /mnt # but include the dietpi_userdata directory, if it is no symlink. # To prevent loops, the backup target dir, log and config are excluded internally. + /mnt/dietpi_userdata/ - /mnt/* - /media/ _EOF_ #----------------------------------------------------------------------------- # Run Backup if (( $INPUT == 1 )); then Run_Backup # Run Restore elif (( $INPUT == -1 )); then Run_Restore #----------------------------------------------------------------------------- # Run menu, if interactive elif (( $G_INTERACTIVE )); then until (( $TARGETMENUID < 0 )) do G_TERM_CLEAR if (( $TARGETMENUID == 1 )); then Menu_Set_Directory else Menu_Main fi done # Save settings Write_Settings_File fi #----------------------------------------------------------------------------------- exit $EXIT_CODE #----------------------------------------------------------------------------------- } And ditepi-global: Spoiler #!/bin/bash { #//////////////////////////////////// # DietPi-Globals # #//////////////////////////////////// # Created by Daniel Knight / daniel.knight@dietpi.com / dietpi.com # #//////////////////////////////////// # # Info: # - Provides shared/global DietPi variables and functions for current bash session and DietPi scripts # - CRITICAL: Use local index variables in for/while loops, or unset them afterwards, else havoc: https://github.com/MichaIng/DietPi/issues/1454 # - Sourced/Loaded in interactive bash sessions via /etc/bashrc.d/dietpi.bash # - Sourced/Loaded at start of most DietPi script #//////////////////////////////////// #----------------------------------------------------------------------------------- # Core variables, functions and environment, used at start of most DietPi scripts #----------------------------------------------------------------------------------- # Script/Program name # - Set this in originating script, after loading globals and before calling G_INIT() # - Used in G_EXEC, G_WHIP and G_DIETPI-NOTIFY functions unset -v G_PROGRAM_NAME # Debug mode # - Set G_DEBUG=1 to enable additional debug output for some DietPi scripts and functions # - This variable is not pre-generated but checked via: [[ $G_DEBUG == 1 ]] #[[ $G_DEBUG == [01] ] || G_DEBUG=0 # Non-interactive mode # - Set G_INTERACTIVE=0 to disable interactive G_EXEC and G_WHIP dialogues # - Set G_INTERACTIVE=1 to force interactive G_EXEC and G_WHIP dialogues # - Default is based on whether STDIN is attached to an open terminal or not: [[ -t 0 ]] # OK | systemd = [[ -t 0 ]] is false # OK | Cron = [[ -t 0 ]] is false # NB | /etc/profile, ~/.profile, /etc/profile.d/, /etc/bash.bashrc, ~/.bashrc and /etc/bashrc.d/ are usually interactive since those are sourced from originating shell/bash session. if [[ $G_INTERACTIVE != [01] ]]; then [[ -t 0 ]] && G_INTERACTIVE=1 || G_INTERACTIVE=0 fi # Disable DietPi-Services # - Set G_DIETPI_SERVICES_DISABLE=1 to disable DietPi-Services # - This variable is not pre-generated but checked via: [[ $G_DIETPI_SERVICES_DISABLE == 1 ]] #[[ $G_DIETPI_SERVICES_DISABLE == [01] ]] || G_DIETPI_SERVICES_DISABLE=0 # DietPi first boot setup stage: -2 = PREP_SYSTEM/Unknown | -1 = 1st boot | 0 = 1st run dietpi-update | 1 = 1st run dietpi-software | 2 = completed | 10 = Pre-installed image, converts to 2 during 1st boot [[ -f '/boot/dietpi/.install_stage' ]] && G_DIETPI_INSTALL_STAGE=$(</boot/dietpi/.install_stage) || G_DIETPI_INSTALL_STAGE=-2 # Hardware details [[ -f '/boot/dietpi/.hw_model' ]] && . /boot/dietpi/.hw_model # DietPi version and Git branch [[ -f '/boot/dietpi/.version' ]] && . /boot/dietpi/.version # - Assign defaults/code version as fallback [[ $G_DIETPI_VERSION_CORE ]] || G_DIETPI_VERSION_CORE=8 [[ $G_DIETPI_VERSION_SUB ]] || G_DIETPI_VERSION_SUB=2 [[ $G_DIETPI_VERSION_RC ]] || G_DIETPI_VERSION_RC=2 [[ $G_GITBRANCH ]] || G_GITBRANCH='master' [[ $G_GITOWNER ]] || G_GITOWNER='MichaIng' # - Save current version and Git branch G_VERSIONDB_SAVE(){ echo "G_DIETPI_VERSION_CORE=$G_DIETPI_VERSION_CORE G_DIETPI_VERSION_SUB=$G_DIETPI_VERSION_SUB G_DIETPI_VERSION_RC=$G_DIETPI_VERSION_RC G_GITBRANCH='$G_GITBRANCH' G_GITOWNER='$G_GITOWNER'" > /boot/dietpi/.version } # Init function for originating script # - Stuff we can't init in main globals/funcs due to /etc/bashrc.d/dietpi.bash load into interactive bash sessions. # - Optional environment variables: # G_INIT_ALLOW_CONCURRENT=1 = Allow concurrent DietPi script execution (default: 0) # G_INIT_WAIT_CONCURRENT=<int> = Max time to wait for concurrent execution to exit before user prompt (default: 5) G_INIT(){ # Set locale to prevent incorrect scraping due to translated command outputs # Set PATH to expected default to rule out issues due to broken environment, e.g. in combination with "su" or "sudo -E" export LC_ALL='C.UTF-8' LANG='C.UTF-8' PATH='/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin' # Set G_PROGRAM_NAME to originating script file (or shell executable) name if it was not set by originating script [[ $G_PROGRAM_NAME ]] || readonly G_PROGRAM_NAME=${0##*/} # HIERARCHY system for G_DIETPI-NOTIFY 3 to reduce highlight or sub script output # shellcheck disable=SC2015 [[ $HIERARCHY =~ ^[0-9]+$ ]] && export HIERARCHY=$((HIERARCHY+1)) || export HIERARCHY=0 # Concurrent execution handling local i=0 if [[ $G_INIT_ALLOW_CONCURRENT == 1 ]] then # Concurrency allowed: Use next free suffix for working directory, avoid race condition by checking via "mkdir" success until mkdir "/tmp/${G_PROGRAM_NAME}_$i" &> /dev/null do # If the directory does not exist, its creation failed, probably due to R/O filesystem, hence break loop! [[ -d /tmp/${G_PROGRAM_NAME}_$i ]] || break ((i++)) done readonly G_WORKING_DIR="/tmp/${G_PROGRAM_NAME}_$i" else # Concurrency not allowed: Use existing working directory as flag local limit=${G_INIT_WAIT_CONCURRENT:-5} while [[ -d /tmp/$G_PROGRAM_NAME ]] do if (( $i < $limit )) then ((i++)) G_DIETPI-NOTIFY 2 "Concurrent execution of $G_PROGRAM_NAME detected, retrying... ($i/$limit)" sleep 1 else G_WHIP_BUTTON_OK_TEXT='Retry' # shellcheck disable=SC2009 G_WHIP_YESNO "WARNING: Concurrent execution of $G_PROGRAM_NAME detected\n Please check if one of the following applies: - This script already runs on another terminal/SSH session. - Currently a cron or systemd background job executes the script. - You started this script from within another DietPi program, causing a loop.\n Please assure that the concurrent execution has finished, before retrying, otherwise cancel this instance.\n The following info might help: $(ps f -eo pid,user,tty,cmd | grep -i '[d]ietpi')" && continue G_DIETPI-NOTIFY 1 "Cancelled $G_PROGRAM_NAME due to concurrent execution" exit 1 fi done readonly G_WORKING_DIR="/tmp/$G_PROGRAM_NAME" fi # Declare exit trap which runs on EXIT signals, including SIGINT and SIGTERM but not SIGKILL! G_EXIT(){ # Execute custom exit function if declared declare -F G_EXIT_CUSTOM &> /dev/null && G_EXIT_CUSTOM # Navigate to /tmp before removing working directory cd /tmp || G_DIETPI-NOTIFY 1 'Failed to navigate to /tmp' # Purge working directory if existent [[ ! -d $G_WORKING_DIR ]] || rm -R "$G_WORKING_DIR" || G_DIETPI-NOTIFY 1 "Failed to remove scripts working directory: $G_WORKING_DIR" } trap 'G_EXIT' EXIT # Create and navigate to scripts working directory or users home if available: https://github.com/MichaIng/DietPi/issues/905#issuecomment-298223705 mkdir -p "$G_WORKING_DIR" && cd "$G_WORKING_DIR" && return G_DIETPI-NOTIFY 1 "Failed to create or enter scripts working directory: $G_WORKING_DIR" if [[ $HOME && -d $HOME ]] then cd "$HOME" && { G_DIETPI-NOTIFY 2 "Entered users home directory: $HOME"; return; } G_DIETPI-NOTIFY 1 "Failed to enter users home directory: $HOME" fi G_DIETPI-NOTIFY 2 "Will stay in current directory: $PWD" } # Clear terminal by moving content into scrollback buffer: https://github.com/MichaIng/DietPi/issues/1615 G_TERM_CLEAR(){ # Without an input terminal, there is no point in doing this. [[ -t 0 ]] || return # Printing terminal height - 1 newlines seems to be the fastest method that is compatible with all terminal types. local lines=$(tput lines) i newlines for ((i=1;i<${lines% *};i++)); do newlines+='\n'; done echo -ne "\e[0m$newlines\e[H" } # DietPi-Notify # $1: # -2 = Processing # $2+ = message # -1 = Autodetect ok or failed # $2 = exit code # $3+ = message # 0 = Ok # $2+ = message # 1 = Failed # $2+ = message # 2 = Info # $2+ = message # 3 = Header # $2 = program name # $3+ = message, prefixed with "${G_NOTIFY_3_MODE}: ", defaults to "Mode: " G_DIETPI-NOTIFY(){ local i ainput_string=("$@") output_string grey green red reset yellow dietpi_green # If this is a terminal, it understands ANSI escape sequences, so use colour, always start left-aligned with colour reset and clear screen from cursor to end. # - Assume if STDIN is a terminal that STDOUT is one as well, e.g. covered by "tee" if [[ -t 0 || -t 1 ]] then output_string='\e[0m\r\e[J' grey='\e[90m' green='\e[32m' red='\e[31m' reset='\e[0m' yellow='\e[33m' dietpi_green='\e[38;5;154m' # Kill existing process animation if this is not a processing message if [[ $1 != '-2' && -w '/tmp/dietpi-process.pid' ]] then kill -9 "$(</tmp/dietpi-process.pid)" &> /dev/null rm -f /tmp/dietpi-process.pid &> /dev/null fi # Else remove all colour codes from input string else shopt -s extglob for i in "${!ainput_string[@]}" do ainput_string[$i]=${ainput_string[$i]//\\e[[0-9]*([;0-9])m} done shopt -u extglob fi local bracket_l="${grey}[$reset" bracket_r="$grey]$reset" local ok="$bracket_l$green OK $bracket_r " failed="$bracket_l${red}FAILED$bracket_r " # Print input array from index $1 Print(){ [[ $1 == 1 && $G_PROGRAM_NAME ]] && output_string+="$grey$G_PROGRAM_NAME |$reset " for ((i=$1; i<${#ainput_string[@]}; i++)) do output_string+=${ainput_string[$i]} done echo -ne "$output_string$reset" } #-------------------------------------------------------------------------------------- # Main Loop #-------------------------------------------------------------------------------------- # Autodetect ok or failed # $2 = exit code # $3+ = message # - Use this at end of DietPi scripts, e.g. G_DIETPI-NOTIFY -1 ${EXIT_CODE:=0} if (( $1 == -1 )); then if (( $2 )); then output_string+=$failed ainput_string+=(' | Exited with error\n') else output_string+=$ok ainput_string+=(' | Completed\n') fi Print 2 #-------------------------------------------------------------------------------------- # Processing # $3+ = message # NB: Do not use this with newlines, literally or "\n", as this would cause parts of the processing message not being overwritten as intended. elif (( $1 == -2 )); then # If this is a terminal, it understands control codes, so make any next output overwrite the processing message. if [[ ( -t 0 || -t 1 ) && $TERM != 'dumb' ]] then # Calculate the amount of output lines and in case move cursor up for correct animation position and to allow overwriting the whole output. local input_string="${G_PROGRAM_NAME:+$G_PROGRAM_NAME | }$*" # - Remove colour codes: Use extended globbing shopt -s extglob input_string=${input_string//\\e[[0-9]*([;0-9])m} shopt -u extglob local screen_width=$(tput cols) local output_lines=$(( ( ${#input_string} + 5 ) / $screen_width )) # +5 = [ .... ] - $1 (( $output_lines )) && ainput_string+=("\e[${output_lines}A") # If we do not print the animation, move the cursor left as well to allow overwriting the whole first line. [[ -t 0 ]] || ainput_string+=('\r') # Else, we add a newline to leave processing message complete. else ainput_string+=('\n') fi # Print animation only if this is the terminal control process as otherwise foreign output might cause a mess and we might be not able to kill the animation. if [[ -t 0 && $TERM != 'dumb' ]] then output_string+="$bracket_l $bracket_r " Print 1 # If redirect to existent PID file fails due to noclobber, don't start processing animation. # - This method prevents a tiny condition race from checking file existence until creating it, when doing: [[ ! -e file ]] && > file set -C if { > /tmp/dietpi-process.pid; } &> /dev/null then set +C Start_Process_Animation(){ local bright_dot='\e[1;33m.' local dimmed_dot='\e[0;33m.' # Alternative: \u23F9 local aprocess_string=( "$bright_dot " "$dimmed_dot$bright_dot " " $dimmed_dot$bright_dot " " $dimmed_dot$bright_dot " " $dimmed_dot$bright_dot " " $dimmed_dot$bright_dot" " $bright_dot" " $bright_dot$dimmed_dot" " $bright_dot$dimmed_dot " " $bright_dot$dimmed_dot " " $bright_dot$dimmed_dot " "$bright_dot$dimmed_dot " ) for (( i=0; i<=${#aprocess_string[@]}; i++ )) do (( i == ${#aprocess_string[@]} )) && i=0 [[ -w '/tmp/dietpi-process.pid' ]] && echo -ne "\e[2G${aprocess_string[$i]}$reset\e[C" || return sleep 0.15 done } { Start_Process_Animation & echo $! > /tmp/dietpi-process.pid; disown; } 2> /dev/null unset -f Start_Process_Animation else set +C fi else output_string+="$bracket_l $yellow.... $bracket_r " Print 1 fi #-------------------------------------------------------------------------------------- # Ok # $2+ = message elif (( $1 == 0 )); then output_string+=$ok ainput_string+=('\n') Print 1 #-------------------------------------------------------------------------------------- # Failed # $2+ = message elif (( $1 == 1 )); then output_string+=$failed ainput_string+=('\n') # Print error messages to STDERR Print 1 >&2 #-------------------------------------------------------------------------------------- # Info # $2+ = message elif (( $1 == 2 )); then output_string+="$bracket_l INFO $bracket_r " # Keep info messages in grey, even if "$G_PROGRAM_NAME | \e[0m" is added: ainput_string[1]="$grey${ainput_string[1]}" ainput_string+=('\n') Print 1 #-------------------------------------------------------------------------------------- # Header # $2 = program name # $3+ = message, prefixed with "${G_NOTIFY_3_MODE}: ", defaults to "Mode: " elif (( $1 == 3 )); then if disable_error=1 G_CHECK_VALIDINT "$HIERARCHY" 1; then local status_subfunction="$HIERARCHY " # > 9 should never occur, however, if it is, lets make it line up regardless (( $HIERARCHY > 9 )) && status_subfunction=$HIERARCHY output_string+="$bracket_l$yellow SUB$status_subfunction$bracket_r $2 > " ainput_string+=('\n') else output_string+=" $dietpi_green$2$reset $grey───────────────────────────────────────────────────── ${G_NOTIFY_3_MODE:-Mode}:$reset " ainput_string+=('\n\n') fi Print 2 fi #----------------------------------------------------------------------------------- # Unset internal functions, otherwise they are accessible from terminal unset -f Print #----------------------------------------------------------------------------------- } # $1 = mode # 2 = Silent check, only returning error code if non-root # 1 = Kill current script only, excluding the shell. # else = Exit all linked scripts (kill all) G_CHECK_ROOT_USER(){ (( $UID )) || return 0 [[ $1 == 2 ]] && return 1 G_DIETPI-NOTIFY 1 'Root privileges required. Please run the command with "sudo" or "G_SUDO".' if [[ $1 == 1 ]] then kill -INT $$ else exit 1 fi } G_CHECK_ROOTFS_RW(){ [[ $G_CHECK_ROOTFS_RW_VERIFIED == 1 ]] && return 0 if grep -q '[[:blank:]]/[[:blank:]].*[[:blank:]]ro,' /proc/mounts then G_DIETPI-NOTIFY 1 'RootFS is currently Read Only (R/O) mounted. Aborting...' G_DIETPI-NOTIFY 2 'DietPi requires RootFS to be Read/Write (R/W) mounted. Please run "dietpi-drive_manager" to re-enable.' exit 1 else export G_CHECK_ROOTFS_RW_VERIFIED=1 fi } #----------------------------------------------------------------------------------- # Shortcut functions #----------------------------------------------------------------------------------- # sudo wrapper that ensures DietPi-Globals with G_* commands are loaded G_SUDO(){ local input=$*; sudo bash -c ". /boot/dietpi/func/dietpi-globals && $input"; } # ownCloud/Nextcloud CLI: Implemented as functions to make then available in scripts [[ -f '/var/www/owncloud/occ' ]] && occ(){ sudo -u www-data php /var/www/owncloud/occ "$@"; } [[ -f '/var/www/nextcloud/occ' ]] && ncc(){ sudo -u www-data php /var/www/nextcloud/occ "$@"; } #----------------------------------------------------------------------------------- # Whiptail (Whippy-da-whip-whip-whip tail!) # - Automatically detects/processes for G_INTERACTIVE #----------------------------------------------------------------------------------- # Input: # - G_WHIP_DEFAULT_ITEM | Optional, to set the default selected/menu item or inputbox entry # - G_WHIP_SIZE_X_MAX=50 | Optional, limits width [in chars], if below available screen width # - G_WHIP_BUTTON_OK_TEXT | Optional, change as needed, defaults to "Ok" # - G_WHIP_BUTTON_CANCEL_TEXT | Optional, change as needed, defaults to "Cancel" # - G_WHIP_MENU_ARRAY | Required for G_WHIP_MENU to set available menu entries, 2 array indices per line: ('item' 'description') # - G_WHIP_CHECKLIST_ARRAY | Required for G_WHIP_CHECKLIST set available checklist options, 3 array indices per line: ('item' 'description' 'on'/'off') # Output: # - G_WHIP_RETURNED_VALUE | Returned value from inputbox/menu/checklist based whiptail items # G_WHIP_DESTROY | Clear vars after run of whiptail G_WHIP_DESTROY(){ unset -v G_WHIP_DEFAULT_ITEM G_WHIP_SIZE_X_MAX G_WHIP_BUTTON_OK_TEXT G_WHIP_BUTTON_CANCEL_TEXT G_WHIP_MENU_ARRAY G_WHIP_CHECKLIST_ARRAY; } # Run once, to be failsafe in case any exported/environment variables are left from originating shell G_WHIP_DESTROY # G_WHIP_INIT # - Update target whiptail size, based on current screen dimensions # - $1 = input mode | 2: Z=G_WHIP_MENU_ARRAY 3: Z=G_WHIP_CHECKLIST_ARRAY G_WHIP_INIT(){ # Automagically set size of whiptail box and contents according to screen size and whiptail type local input_mode=$1 # Update backtitle WHIP_BACKTITLE=$G_HW_MODEL_NAME local active_ip=$(G_GET_NET -q ip) [[ $active_ip ]] && WHIP_BACKTITLE+=" | IP: $active_ip" # Set default button text, if not defined G_WHIP_BUTTON_OK_TEXT=${G_WHIP_BUTTON_OK_TEXT:-Ok} G_WHIP_BUTTON_CANCEL_TEXT=${G_WHIP_BUTTON_CANCEL_TEXT:-Cancel} # Get current screen dimensions WHIP_SIZE_X=$(tput cols) WHIP_SIZE_Y=$(tput lines) # - Limit and reset non-valid integer values to 120 characters per line (( $WHIP_SIZE_X <= 120 )) || WHIP_SIZE_X=120 # - If width is below 9 characters, the text field starts to cover the internal margin, regardless of content or button text, hence 9 is the absolute minimum. (( $WHIP_SIZE_X >= 9 )) || WHIP_SIZE_X=9 # - G_WHIP_SIZE_X_MAX allows to further reduce width, e.g. to keep X/Y ratio in beautiful range. disable_error=1 G_CHECK_VALIDINT "$G_WHIP_SIZE_X_MAX" 0 $WHIP_SIZE_X && WHIP_SIZE_X=$G_WHIP_SIZE_X_MAX # - If height is below 7 lines, not a single line of text can be shown, hence 7 is the reasonable minimum. (( $WHIP_SIZE_Y >= 7 )) || WHIP_SIZE_Y=7 # Calculate lines required to show all text content local whip_lines_text=6 # Due to internal margins, the available height is 6 lines smaller local whip_chars_text=$(( $WHIP_SIZE_X - 4 )) # Due to internal margins, the available width is 4 characters smaller WHIP_SCROLLTEXT= # Add "--scrolltext" automatically if text height exceeds max available Process_Line(){ local split line=$1 # Split line by "\n" newline escape sequences, the only one which is interpreted by whiptail, in a strict way: "\\n" still creates a newline, hence the sequence cannot be escaped! while [[ $line == *'\n'* ]] do # Grab first line split=${line%%\\n*} # Add required line + additional lines due to automated line breaks, if text exceeds internal box (( whip_lines_text += 1 + ( ${#split} - 1 ) / $whip_chars_text )) # Stop counting if required size exceeds screen already (( $whip_lines_text > $WHIP_SIZE_Y )) && return 1 # Cut away handled line from string line=${line#*\\n} done # Process remaining line (( whip_lines_text += 1 + ( ${#line} - 1 ) / $whip_chars_text )) # Stop counting if required size exceeds screen already (( $whip_lines_text <= $WHIP_SIZE_Y )) || return 1 } # - WHIP_MESSAGE if [[ $WHIP_ERROR$WHIP_MESSAGE ]]; then while read -r line; do Process_Line "$line" || break; done <<< "$WHIP_ERROR$WHIP_MESSAGE" # - WHIP_TEXTFILE elif [[ $WHIP_TEXTFILE ]]; then while read -r line; do Process_Line "$line" || break; done < "$WHIP_TEXTFILE" fi unset -f Process_Line # Process menu and checklist # - G_WHIP_MENU if [[ $input_mode == 2 ]]; then # Requires 1 additional line for text ((whip_lines_text++)) # Lines required for menu: ( ${#array} + 1 ) to round up on uneven array entries WHIP_SIZE_Z=$(( ( ${#G_WHIP_MENU_ARRAY[@]} + 1 ) / 2 )) # Auto length for ─ # - Get max length of all lines in array indices 1 + 2n | '' 'this one' local i character_count_max=0 for (( i=1; i<${#G_WHIP_MENU_ARRAY[@]}; i+=2 )) do (( ${#G_WHIP_MENU_ARRAY[$i]} > $character_count_max )) && character_count_max=${#G_WHIP_MENU_ARRAY[$i]} done ((character_count_max--)) # -1 for additional ● # - Now add the additional required lines for (( i=1; i<${#G_WHIP_MENU_ARRAY[@]}; i+=2 )) do [[ ${G_WHIP_MENU_ARRAY[$i]} == '●'* ]] || continue while (( ${#G_WHIP_MENU_ARRAY[$i]} < $character_count_max )) do G_WHIP_MENU_ARRAY[$i]+='─' done G_WHIP_MENU_ARRAY[$i]+='●' done # - G_WHIP_CHECKLIST elif [[ $input_mode == 3 ]]; then # Lines required for checklist: ( ${#array} + 2 ) to round up single+double array entries WHIP_SIZE_Z=$(( ( ${#G_WHIP_CHECKLIST_ARRAY[@]} + 2 ) / 3 )) # Auto length for ─ # - Get max length of all lines in array indices 1 + 3n 1st | '' 'this one' '' local i character_count_max=0 for (( i=1; i<${#G_WHIP_CHECKLIST_ARRAY[@]}; i+=3 )) do (( ${#G_WHIP_CHECKLIST_ARRAY[$i]} > $character_count_max )) && character_count_max=${#G_WHIP_CHECKLIST_ARRAY[$i]} done ((character_count_max--)) # -1 for additional ● # - Now add the additional required lines for (( i=1; i<${#G_WHIP_CHECKLIST_ARRAY[@]}; i+=3 )) do [[ ${G_WHIP_CHECKLIST_ARRAY[$i]} == '●'* ]] || continue while (( ${#G_WHIP_CHECKLIST_ARRAY[$i]} < $character_count_max )) do G_WHIP_CHECKLIST_ARRAY[$i]+='─' done G_WHIP_CHECKLIST_ARRAY[$i]+='●' done fi # Adjust sizes to fit content # - G_WHIP_MENU/G_WHIP_CHECKLIST needs to hold text + selection field (WHIP_SIZE_Z) if [[ $input_mode == [23] ]]; then # If required lines would exceed screen, reduce WHIP_SIZE_Z if (( $whip_lines_text + $WHIP_SIZE_Z > $WHIP_SIZE_Y )); then WHIP_SIZE_Z=$(( $WHIP_SIZE_Y - $whip_lines_text )) # Assure at least 2 lines to have the selection field scroll bar identifiable if (( $WHIP_SIZE_Z < 2 )); then WHIP_SIZE_Z=2 # Since text is partly hidden now, add text scroll ability and info to backtitle WHIP_SCROLLTEXT='--scrolltext' WHIP_BACKTITLE+=' | Use up/down buttons to scroll text' fi # else reduce WHIP_SIZE_Y to hold all content else WHIP_SIZE_Y=$(( $whip_lines_text + $WHIP_SIZE_Z )) fi # - Everything else needs to hold text only elif (( $whip_lines_text > $WHIP_SIZE_Y )); then WHIP_SCROLLTEXT='--scrolltext' WHIP_BACKTITLE+=' | Use up/down buttons to scroll text' else WHIP_SIZE_Y=$whip_lines_text fi } # G_WHIP_MSG "message" # - Display a message from input string G_WHIP_MSG(){ local WHIP_MESSAGE=$* if [[ $G_INTERACTIVE == 1 ]]; then local WHIP_ERROR WHIP_BACKTITLE WHIP_SCROLLTEXT WHIP_SIZE_X WHIP_SIZE_Y G_WHIP_INIT whiptail ${G_PROGRAM_NAME:+--title "$G_PROGRAM_NAME"} --backtitle "$WHIP_BACKTITLE" --msgbox "$WHIP_MESSAGE" --ok-button "$G_WHIP_BUTTON_OK_TEXT" $WHIP_SCROLLTEXT "$WHIP_SIZE_Y" "$WHIP_SIZE_X" else G_DIETPI-NOTIFY 2 "$WHIP_MESSAGE" fi G_WHIP_DESTROY } # G_WHIP_VIEWFILE "/path/to/file" # - Display content from input file # - Exit code: 1=file not found, else=file shown or noninteractive G_WHIP_VIEWFILE(){ local result=0 if [[ $G_INTERACTIVE == 1 ]]; then local WHIP_ERROR WHIP_MESSAGE WHIP_BACKTITLE WHIP_SCROLLTEXT WHIP_SIZE_X WHIP_SIZE_Y WHIP_TEXTFILE=$1 header='File viewer' [[ $log == 1 ]] && header='Log viewer' if [[ -f $WHIP_TEXTFILE ]]; then G_WHIP_INIT whiptail --title "${G_PROGRAM_NAME:+$G_PROGRAM_NAME | }$header" --backtitle "$WHIP_BACKTITLE" --textbox "$WHIP_TEXTFILE" --ok-button "$G_WHIP_BUTTON_OK_TEXT" $WHIP_SCROLLTEXT "$WHIP_SIZE_Y" "$WHIP_SIZE_X" else result=1 WHIP_ERROR="[FAILED] File does not exist: $WHIP_TEXTFILE" G_WHIP_INIT whiptail --title "${G_PROGRAM_NAME:+$G_PROGRAM_NAME | }$header" --backtitle "$WHIP_BACKTITLE" --msgbox "$WHIP_ERROR" --ok-button "$G_WHIP_BUTTON_OK_TEXT" $WHIP_SCROLLTEXT "$WHIP_SIZE_Y" "$WHIP_SIZE_X" fi fi G_WHIP_DESTROY return $result } # G_WHIP_YESNO "message" # - Prompt user for Yes/No | Ok/Cancel choice and return result # - Exit code: 0=Yes/Ok, else=No/Cancel or noninteractive G_WHIP_YESNO(){ local result=1 default_no='--defaultno' [[ ${G_WHIP_DEFAULT_ITEM,,} == 'yes' || ${G_WHIP_DEFAULT_ITEM,,} == 'ok' ]] && result=0 default_no= if [[ $G_INTERACTIVE == 1 ]]; then local WHIP_ERROR WHIP_BACKTITLE WHIP_SCROLLTEXT WHIP_SIZE_X WHIP_SIZE_Y WHIP_MESSAGE=$* G_WHIP_INIT whiptail ${G_PROGRAM_NAME:+--title "$G_PROGRAM_NAME"} --backtitle "$WHIP_BACKTITLE" --yesno "$WHIP_MESSAGE" --yes-button "$G_WHIP_BUTTON_OK_TEXT" --no-button "$G_WHIP_BUTTON_CANCEL_TEXT" "$default_no" $WHIP_SCROLLTEXT "$WHIP_SIZE_Y" "$WHIP_SIZE_X" result=$? fi G_WHIP_DESTROY return $result } # G_WHIP_INPUTBOX "message" # - Prompt user to input text and save it to G_WHIP_RETURNED_VALUE # - Exit code: 0=input done, else=user cancelled or noninteractive G_WHIP_INPUTBOX(){ local result=1 unset -v G_WHIP_RETURNED_VALUE # in case left from last G_WHIP if [[ $G_INTERACTIVE == 1 ]]; then local WHIP_ERROR WHIP_BACKTITLE WHIP_SCROLLTEXT WHIP_SIZE_X WHIP_SIZE_Y WHIP_MESSAGE=$* while : do G_WHIP_INIT G_WHIP_RETURNED_VALUE=$(whiptail ${G_PROGRAM_NAME:+--title "$G_PROGRAM_NAME"} --backtitle "$WHIP_BACKTITLE" --inputbox "$WHIP_ERROR$WHIP_MESSAGE" --ok-button "$G_WHIP_BUTTON_OK_TEXT" --cancel-button "$G_WHIP_BUTTON_CANCEL_TEXT" $WHIP_SCROLLTEXT "$WHIP_SIZE_Y" "$WHIP_SIZE_X" "$G_WHIP_DEFAULT_ITEM" 3>&1 1>&2 2>&3-; echo $? > /tmp/.G_WHIP_INPUTBOX_RESULT) result=$(</tmp/.G_WHIP_INPUTBOX_RESULT); rm -f /tmp/.G_WHIP_INPUTBOX_RESULT [[ $result == 0 && -z $G_WHIP_RETURNED_VALUE ]] && { WHIP_ERROR='[FAILED] An input value was not entered, please try again...\n\n'; continue; } break done fi G_WHIP_DESTROY return "$result" } # G_WHIP_PASSWORD "message" # - Prompt user to input password and save it in variable "result" # - Originating script must "unset result" after value has been handled for security reasons! # - Exit code: 0=input done + passwords match, else=noninteractive (Cancelling is disabled since no password in originating script can cause havoc!) G_WHIP_PASSWORD(){ local return_value=1 unset -v result # in case left from last call if [[ $G_INTERACTIVE == 1 ]]; then local WHIP_ERROR WHIP_BACKTITLE WHIP_SCROLLTEXT WHIP_SIZE_X WHIP_SIZE_Y WHIP_MESSAGE=$* while : do G_WHIP_INIT local password_0=$(whiptail ${G_PROGRAM_NAME:+--title "$G_PROGRAM_NAME"} --backtitle "$WHIP_BACKTITLE" --passwordbox "$WHIP_ERROR$WHIP_MESSAGE" --ok-button "$G_WHIP_BUTTON_OK_TEXT" --nocancel $WHIP_SCROLLTEXT "$WHIP_SIZE_Y" "$WHIP_SIZE_X" 3>&1 1>&2 2>&3-) [[ $password_0 ]] || { WHIP_ERROR='[FAILED] No input made, please try again...\n\n'; continue; } local password_1=$(whiptail ${G_PROGRAM_NAME:+--title "$G_PROGRAM_NAME"} --backtitle "$WHIP_BACKTITLE" --passwordbox 'Please retype and confirm your input:' --ok-button "$G_WHIP_BUTTON_OK_TEXT" --nocancel 7 "$WHIP_SIZE_X" 3>&1 1>&2 2>&3-) [[ $password_0 == "$password_1" ]] || { WHIP_ERROR='[FAILED] Inputs do not match, please try again...\n\n'; continue; } result=$password_0 return_value=0 break done fi G_WHIP_DESTROY return $return_value } # G_WHIP_MENU "message" # - Prompt user to select option from G_WHIP_MENU_ARRAY and save choice to G_WHIP_RETURNED_VALUE # - Exit code: 0=selection done, else=user cancelled or noninteractive G_WHIP_MENU() { local result=1 unset -v G_WHIP_RETURNED_VALUE # in case left from last call [[ $G_INTERACTIVE == 1 ]] && until [[ $G_WHIP_RETURNED_VALUE ]] # Stay in menu if empty option was selected (separator line) do local WHIP_ERROR WHIP_BACKTITLE WHIP_SCROLLTEXT WHIP_SIZE_X WHIP_SIZE_Y WHIP_SIZE_Z WHIP_MESSAGE=$* G_WHIP_INIT 2 G_WHIP_RETURNED_VALUE=$(whiptail ${G_PROGRAM_NAME:+--title "$G_PROGRAM_NAME"} --backtitle "$WHIP_BACKTITLE" --menu "$WHIP_MESSAGE" --ok-button "$G_WHIP_BUTTON_OK_TEXT" --cancel-button "$G_WHIP_BUTTON_CANCEL_TEXT" --default-item "$G_WHIP_DEFAULT_ITEM" $WHIP_SCROLLTEXT "$WHIP_SIZE_Y" "$WHIP_SIZE_X" $WHIP_SIZE_Z "${G_WHIP_MENU_ARRAY[@]}" 3>&1 1>&2 2>&3-; echo $? > /tmp/.WHIP_MENU_RESULT) result=$(</tmp/.WHIP_MENU_RESULT); rm -f /tmp/.WHIP_MENU_RESULT [[ ${result:=1} == 0 ]] || break # Exit loop in case of cancel button selection or error or if .WHIP_MENU_RESULT could not be created done G_WHIP_DESTROY return "$result" } # G_WHIP_CHECKLIST "message" # - Prompt user to select multiple options from G_WHIP_CHECKLIST_ARRAY and save choice to G_WHIP_RETURNED_VALUE # - Exit code: 0=selection done, else=user cancelled or noninteractive G_WHIP_CHECKLIST() { local result=1 unset -v G_WHIP_RETURNED_VALUE # in case left from last call if [[ $G_INTERACTIVE == 1 ]] then local WHIP_ERROR WHIP_BACKTITLE WHIP_SCROLLTEXT WHIP_SIZE_X WHIP_SIZE_Y WHIP_SIZE_Z WHIP_MESSAGE=$* G_WHIP_INIT 3 G_WHIP_RETURNED_VALUE=$(whiptail ${G_PROGRAM_NAME:+--title "$G_PROGRAM_NAME"} --backtitle "$WHIP_BACKTITLE | Use spacebar to toggle selection" --checklist "$WHIP_MESSAGE" --separate-output --ok-button "$G_WHIP_BUTTON_OK_TEXT" --cancel-button "$G_WHIP_BUTTON_CANCEL_TEXT" --default-item "$G_WHIP_DEFAULT_ITEM" $WHIP_SCROLLTEXT "$WHIP_SIZE_Y" "$WHIP_SIZE_X" $WHIP_SIZE_Z "${G_WHIP_CHECKLIST_ARRAY[@]}" 3>&1 1>&2 2>&3-; echo $? > /tmp/.WHIP_CHECKLIST_RESULT) G_WHIP_RETURNED_VALUE=$(echo -e "$G_WHIP_RETURNED_VALUE" | tr '\n' ' ') result=$(</tmp/.WHIP_CHECKLIST_RESULT); rm -f /tmp/.WHIP_CHECKLIST_RESULT fi G_WHIP_DESTROY return "${result:-1}" } #----------------------------------------------------------------------------------- # Error handled command execution wrapper #----------------------------------------------------------------------------------- # IMPORTANT: # - Never pipe G_EXEC! "G_EXEC command | command" leads to G_EXEC not being able to unset G_EXEC_* variables and functions from originating shell or kill the originating script in case of error. # Required input: # - $@=<command> | Command to execute # Optional input: # - $G_EXEC_DESC=<text> | Command description to print instead of raw command string # - $G_EXEC_RETRIES=<int> | Amount of non-interactive retries in case of error, before doing interactive error prompt # - G_EXEC_PRE_FUNC(){} | Function to call before every input command attempt, e.g. to re-evaluate variables # - G_EXEC_POST_FUNC(){} | Function to call after every input command attempt, e.g. to handle errors without error exit code # - $G_EXEC_OUTPUT=1 | Print full command output instead of animated processing message # - $G_EXEC_OUTPUT_COL='\e[90m' | Override colour of command output via console colour code, requires $G_EXEC_OUTPUT=1 # - $G_EXEC_NOFAIL=1 | On error, override as success, only useful to replace verbose output by animated processing message, inherits $G_EXEC_NOHALT=1 and $G_EXEC_NOEXIT=1 # - $G_EXEC_NOHALT=1 | On error, print short error message only, skip error handler menu and do not exit script, inherits $G_EXEC_NOEXIT=1 # - $G_EXEC_NOEXIT=1 | On error, do not exit script, inherited by $G_EXEC_NOHALT=1 # - $G_EXEC_ARRAY_TEXT[] | Add additional entries to error handler menu # - $G_ECEC_ARRAY_ACTION[] | Associative array, containing uneven $G_EXEC_ARRAY_TEXT[] values as keys and related commands as values G_EXEC(){ local exit_code fp_log='/tmp/G_EXEC_LOG' attempt=1 acommand=("$@") ecommand=${*//\\/\\\\} # Enter retry loop while : do declare -F G_EXEC_PRE_FUNC &> /dev/null && G_EXEC_PRE_FUNC # Execute command, store output to $fp_log file and store exit code to $exit_code variable # - Print full command output if $G_EXEC_OUTPUT=1 is given if [[ $G_EXEC_OUTPUT == 1 ]]; then # Print $G_EXEC_DESC if given, else raw input command string and show current non-interactive attempt count if $G_EXEC_RETRIES is given G_DIETPI-NOTIFY 2 "${G_EXEC_DESC:-$ecommand}, please wait...${G_EXEC_RETRIES:+ ($attempt/$((G_EXEC_RETRIES+1)))}" [[ $G_EXEC_OUTPUT_COL ]] && echo -ne "$G_EXEC_OUTPUT_COL" "${acommand[@]}" 2>&1 | tee $fp_log exit_code=${PIPESTATUS[0]} [[ $G_EXEC_OUTPUT_COL ]] && echo -ne '\e[0m' # - Else print animated processing message only else G_DIETPI-NOTIFY -2 "${G_EXEC_DESC:-$ecommand}${G_EXEC_RETRIES:+ ($attempt/$((G_EXEC_RETRIES+1)))}" "${acommand[@]}" &> $fp_log exit_code=$? fi declare -F G_EXEC_POST_FUNC &> /dev/null && G_EXEC_POST_FUNC # Override exit code if $G_EXEC_NOFAIL=1 is given [[ $G_EXEC_NOFAIL == 1 ]] && exit_code=0 ### Success: Print OK and exit retry loop [[ $exit_code == 0 ]] && { G_DIETPI-NOTIFY 0 "${G_EXEC_DESC:-$ecommand}"; break; } ### Error # Retry non-interactively if current $attempt is <= $G_EXEC_RETRIES [[ $attempt -le $G_EXEC_RETRIES ]] && { ((attempt++)) && continue; } # Print FAILED, append raw command string if $G_EXEC_DESC is given G_DIETPI-NOTIFY 1 "${G_EXEC_DESC:+$G_EXEC_DESC\n - Command: }$ecommand" # Exit retry loop if $G_EXEC_NOHALT=1 is given [[ $G_EXEC_NOHALT == 1 ]] && break # Prepare error handler menu and GitHub issue template local fp_error_report='/tmp/G_EXEC_ERROR_REPORT' log_content=$(<$fp_log) image_creator preimage_name dietpi_version="v$G_DIETPI_VERSION_CORE.$G_DIETPI_VERSION_SUB.$G_DIETPI_VERSION_RC ($G_GITOWNER/$G_GITBRANCH)" last_whip_menu_item sent_bug_report if [[ -f '/boot/dietpi/.prep_info' ]]; then image_creator=$(mawk 'NR==1' /boot/dietpi/.prep_info) [[ $image_creator == 0 ]] && image_creator='DietPi Core Team' preimage_name=$(mawk 'NR==2' /boot/dietpi/.prep_info) fi # Create GitHub issue template if error was produced by one of our scripts [[ ${G_PROGRAM_NAME,,} == 'dietpi-'* ]] && echo -e "\e[41m --------------------------------------------------------------------- - DietPi has encountered an error - - Please create a ticket: https://github.com/MichaIng/DietPi/issues - - Copy and paste only the BLUE lines below into the ticket - ---------------------------------------------------------------------\e[44m #### Details: - Date | $(date) - DietPi version | $dietpi_version - Image creator | $image_creator - Pre-image | $preimage_name - Hardware | $G_HW_MODEL_NAME (ID=$G_HW_MODEL) - Kernel version | \`$(uname -a)\` - Distro | $G_DISTRO_NAME (ID=$G_DISTRO${G_RASPBIAN:+,RASPBIAN=$G_RASPBIAN}) - Command | \`$*\` - Exit code | $exit_code - Software title | $G_PROGRAM_NAME #### Steps to reproduce: <!-- Explain how to reproduce the issue --> 1. ... 2. ... #### Expected behaviour: <!-- What SHOULD happen? --> - ... #### Actual behaviour: <!-- What IS happening? --> - ... #### Extra details: <!-- Please post any extra details that might help solve the issue --> - ... #### Additional logs: \`\`\` $log_content \`\`\`\e[41m ---------------------------------------------------------------------\e[0m" > $fp_error_report # Enter error handler menu loop in interactive mode [[ $G_INTERACTIVE == 1 ]] && while : do G_WHIP_MENU_ARRAY=('Retry' ': Re-run the last command that failed') # Add targeted solution suggestions, passed via $G_EXEC_ARRAY_TEXT[] and $G_EXEC_ARRAY_ACTION[${G_EXEC_ARRAY_TEXT[]}] [[ $G_EXEC_ARRAY_TEXT ]] && G_WHIP_MENU_ARRAY+=("${G_EXEC_ARRAY_TEXT[@]}") # Allow to open DietPi-Config if this error was not produced within DietPi-Config pgrep -cf 'dietpi-config' &> /dev/null || G_WHIP_MENU_ARRAY+=('DietPi-Config' ': Edit network, APT/NTP mirror settings etc') G_WHIP_MENU_ARRAY+=('Open subshell' ': Open a subshell to investigate or solve the issue') # Allow to send bug report, if it was produced by one of our scripts [[ ${G_PROGRAM_NAME,,} == 'dietpi-'* && $G_PROGRAM_NAME != 'DietPi-PREP' ]] && G_WHIP_MENU_ARRAY+=('Send report' ': Uploads bugreport containing system info to DietPi') G_WHIP_MENU_ARRAY+=('' '●─ Devs only ') G_WHIP_MENU_ARRAY+=('Change command' ': Adjust and rerun the command') # Show "Ignore" on cancel button if $G_EXEC_NOEXIT=1 is given, else "Exit" [[ $G_EXEC_NOEXIT == 1 ]] && G_WHIP_BUTTON_CANCEL_TEXT='Ignore' || G_WHIP_BUTTON_CANCEL_TEXT='Exit' G_WHIP_DEFAULT_ITEM=${last_whip_menu_item:-Retry} G_WHIP_MENU "${G_EXEC_DESC:+$(mawk '{gsub("\\\e[[0-9][;0-9]*m","");print}' <<< "$G_EXEC_DESC")\n} - Command: $* - Exit code: $exit_code - DietPi version: $dietpi_version | HW_MODEL: $G_HW_MODEL | HW_ARCH: $G_HW_ARCH | DISTRO: $G_DISTRO ${image_creator:+ - Image creator: $image_creator\n}${preimage_name:+ - Pre-image: $preimage_name\n} - Error log: $log_content" || break # Exit error handler menu loop on cancel last_whip_menu_item=$G_WHIP_RETURNED_VALUE if [[ $G_WHIP_RETURNED_VALUE == 'Retry' ]]; then # Reset current $attempt and continue retry loop attempt=1 continue 2 elif [[ $G_WHIP_RETURNED_VALUE == 'DietPi-Config' ]]; then /boot/dietpi/dietpi-config elif [[ $G_WHIP_RETURNED_VALUE == 'Open subshell' ]]; then G_WHIP_MSG 'A bash subshell will now open which allows you to investigate and/or fix the issue. \nPlease use the "exit" command when you are finished, to return to this error handler menu.' # Prevent dietpi-login call in subshell local reallow_dietpi_login=1 [[ $G_DIETPI_LOGIN ]] && reallow_dietpi_login=0 export G_DIETPI_LOGIN=1 bash &> /dev/tty < /dev/tty (( $reallow_dietpi_login )) && unset -v G_DIETPI_LOGIN elif [[ $G_WHIP_RETURNED_VALUE == 'Send report' ]]; then /boot/dietpi/dietpi-bugreport 1 && sent_bug_report=1 read -rp ' Press any key to continue...' elif [[ $G_WHIP_RETURNED_VALUE == 'Change command' ]]; then G_WHIP_DEFAULT_ITEM=$* if G_WHIP_INPUTBOX 'Please enter/alter the command to be executed. \nNB: Please only use this solution if you know for sure that it will not cause follow up issues from the originating script. It will e.g. allow you to continue a certain software install, but if you edit the download link, the originating script might expect files which are not present. \nUse this work caution!'; then G_DIETPI-NOTIFY 2 "Executing alternative command: ${G_WHIP_RETURNED_VALUE//\\/\\\\}" $G_WHIP_RETURNED_VALUE exit_code=$? G_DIETPI-NOTIFY -1 $exit_code 'Alternative command execution' # Exit retry loop if alternative command succeeded, else stay in menu loop and wait for key press to allow reviewing alternative command output # shellcheck disable=SC2015 [[ $exit_code == 0 ]] && break 2 || read -rp 'Press any key to return to error handler menu...' fi # Attempt targeted solution, passed via $G_EXEC_ARRAY_TEXT[] and $G_EXEC_ARRAY_ACTION[${G_EXEC_ARRAY_TEXT[]}] elif [[ $G_WHIP_RETURNED_VALUE ]]; then ${G_EXEC_ARRAY_ACTION[$G_WHIP_RETURNED_VALUE]} read -rp 'Press any key to return to error handler menu...' fi done # Error has not been solved, print GitHub issue template if it was produced and exit error handler menu loop if [[ -f $fp_error_report ]]; then # Add bug report ID if it was sent [[ $sent_bug_report == 1 ]] && sed -i "/^- Date | /a\- Bug report | $G_HW_UUID" $fp_error_report cat $fp_error_report fi break done # Do not exit originating script if $G_EXEC_NOEXIT=1 or $G_EXEC_NOHALT=1 is given local noexit [[ $G_EXEC_NOEXIT == 1 || $G_EXEC_NOHALT == 1 ]] && noexit=1 # Cleanup rm -f $fp_log $fp_error_report unset -v G_EXEC_DESC G_EXEC_RETRIES G_EXEC_OUTPUT G_EXEC_OUTPUT_COL G_EXEC_NOFAIL G_EXEC_NOHALT G_EXEC_NOEXIT G_EXEC_ARRAY_TEXT G_EXEC_ARRAY_ACTION unset -f G_EXEC_PRE_FUNC G_EXEC_POST_FUNC # In case of unresolved error when exiting originating script, inform user and kill via SIGINT to prevent exiting from interactive shell session [[ $exit_code == 0 || $noexit == 1 ]] || { G_DIETPI-NOTIFY 1 "Unable to continue, ${G_PROGRAM_NAME:-command} will now terminate."; kill -INT $$; } # Else return exit code return $exit_code } #----------------------------------------------------------------------------------- # Multithreading handler #----------------------------------------------------------------------------------- # Not yet compatible with dietpi global commands. single bash commands only with no error handling. G_THREAD_START(){ # Run in blocking mode if [[ $G_THREADING_ENABLED == 0 ]]; then G_DIETPI-NOTIFY 2 "G_THREADING disabled, running command in blocking mode | $*" "$@" # Launch as background process else [[ $G_THREAD_COUNT =~ ^[0-9]+$ ]] || G_THREAD_COUNT=-1 ((G_THREAD_COUNT++)) G_THREAD_COMMAND[$G_THREAD_COUNT]=$* # Store for later output with G_THREAD_WAIT echo -1337 > "/tmp/.G_THREAD_EXITCODE_$G_THREAD_COUNT" { { G_INTERACTIVE=0 "$@" &> "/tmp/.G_THREAD_COMMAND_$G_THREAD_COUNT"; echo $? > "/tmp/.G_THREAD_EXITCODE_$G_THREAD_COUNT"; } & disown; } &> /dev/null G_DIETPI-NOTIFY 2 "G_THREAD_START_$G_THREAD_COUNT | $*" fi } G_THREAD_WAIT(){ #local wait_for_specific_thread_pid=-1 #[[ $1 ]] && wait_for_specific_thread_pid=$1 local i waiting_for exit_code # Wait until all threads finished while : do for i in "${!G_THREAD_COMMAND[@]}" do [[ -f /tmp/.G_THREAD_EXITCODE_$i && $(<"/tmp/.G_THREAD_EXITCODE_$i") == '-1337' ]] || continue # Print what we are waiting for, update processing message if thread changed since last loop [[ $waiting_for == "$i" ]] || G_DIETPI-NOTIFY -2 "G_THREAD_WAIT_$i | ${G_THREAD_COMMAND[$i]}" waiting_for=$i sleep 1 continue 2 done break done G_DIETPI-NOTIFY 0 'G_THREAD: All threads finished' # Check all thread's exit codes for issues for i in "${!G_THREAD_COMMAND[@]}" do if [[ -r /tmp/.G_THREAD_EXITCODE_$i ]]; then exit_code=$(<"/tmp/.G_THREAD_EXITCODE_$i") (( $exit_code )) && G_WHIP_MSG "G_THREAD ERROR:\n - Command = ${G_THREAD_COMMAND[$i]}\n - Exit code = $exit_code\n\n$(<"/tmp/.G_THREAD_COMMAND_$i")" else G_DIETPI-NOTIFY 2 "DEBUG: /tmp/.G_THREAD_EXITCODE_$i does not exist or is not readable" fi done rm -f /tmp/.G_THREAD* unset -v G_THREAD_COUNT G_THREAD_COMMAND } #----------------------------------------------------------------------------------- # Network connection checks #----------------------------------------------------------------------------------- # General network connection check # - Checks general network connectivity by pinging a raw IP that must be publicly reachable at all time. # - Uses the given input argument as IP to test against, else CONFIG_CHECK_CONNECTION_IP from dietpi.txt, else defaults to 9.9.9.9 (Quad9 DNS IP). # - Uses G_CHECK_URL_TIMEOUT and G_CHECK_URL_ATTEMPTS variables, else CONFIG_G_CHECK_URL_TIMEOUT + CONFIG_G_CHECK_URL_ATTEMPTS from dietpi.txt, else defaults to 10 seconds and 2 attempts (1 retry). # - Optional arguments: # $* = IP + optional ping arguments # - Optional variables: # G_CHECK_URL_TIMEOUT to override default and dietpi.txt set timeout # G_CHECK_URL_ATTEMPTS to override default and dietpi.txt set attempts G_CHECK_CON(){ # Obtain IP local ip if [[ ! $* ]]; then ip=$(sed -n '/^[[:blank:]]*CONFIG_CHECK_CONNECTION_IP=/{s/^[^=]*=//p;q}' /boot/dietpi.txt) [[ $ip ]] || ip='9.9.9.9' fi # Obtain timeout if ! disable_error=1 G_CHECK_VALIDINT "$G_CHECK_URL_TIMEOUT" 0; then G_CHECK_URL_TIMEOUT=$(sed -n '/^[[:blank:]]*CONFIG_G_CHECK_URL_TIMEOUT=/{s/^[^=]*=//p;q}' /boot/dietpi.txt) disable_error=1 G_CHECK_VALIDINT "$G_CHECK_URL_TIMEOUT" 0 || G_CHECK_URL_TIMEOUT=10 fi # Obtain attempts if ! disable_error=1 G_CHECK_VALIDINT "$G_CHECK_URL_ATTEMPTS" 1; then G_CHECK_URL_ATTEMPTS=$(sed -n '/^[[:blank:]]*CONFIG_G_CHECK_URL_ATTEMPTS=/{s/^[^=]*=//p;q}' /boot/dietpi.txt) disable_error=1 G_CHECK_VALIDINT "$G_CHECK_URL_ATTEMPTS" 1 || G_CHECK_URL_ATTEMPTS=2 fi G_EXEC_RETRIES=$(( $G_CHECK_URL_ATTEMPTS - 1 )) # 2 attempts = 1 retry # Re-assign timeout ($5 => acommand[4]) on every retry, in case it is changed as solution attempt declare -F G_EXEC_PRE_FUNC &> /dev/null && eval "G_EXEC_PRE_FUNC_ORIG()$(declare -f G_EXEC_PRE_FUNC | tail -n +2)" G_EXEC_PRE_FUNC(){ acommand[4]=$G_CHECK_URL_TIMEOUT; declare -F G_EXEC_PRE_FUNC_ORIG &> /dev/null && G_EXEC_PRE_FUNC_ORIG; } G_EXEC_DESC='Checking network connectivity' G_EXEC ping -nc 1 -W $G_CHECK_URL_TIMEOUT ${ip:-"$@"} local exit_code=$? unset -v G_CHECK_URL_TIMEOUT G_CHECK_URL_ATTEMPTS unset -f G_EXEC_PRE_FUNC_ORIG return $exit_code } # General DNS resolver check # - Checks general DNS capabilities by pinging a domain that must be publicly reachable at all time. # - Uses the given input argument as domain to test against, else CONFIG_CHECK_DNS_DOMAIN from dietpi.txt, else defaults to dns9.quad9.net (Quad9 DNS domain). # - Uses G_CHECK_URL_TIMEOUT and G_CHECK_URL_ATTEMPTS variables, else CONFIG_G_CHECK_URL_TIMEOUT + CONFIG_G_CHECK_URL_ATTEMPTS from dietpi.txt, else defaults to 10 seconds and 2 attempts (1 retry). # - Optional arguments: # $* = domain + optional ping arguments # - Optional variables: # G_CHECK_URL_TIMEOUT to override default and dietpi.txt set timeout # G_CHECK_URL_ATTEMPTS to override default and dietpi.txt set attempts G_CHECK_DNS(){ # Obtain IP local domain if [[ ! $* ]]; then domain=$(sed -n '/^[[:blank:]]*CONFIG_CHECK_DNS_DOMAIN=/{s/^[^=]*=//p;q}' /boot/dietpi.txt) [[ $domain ]] || domain='dns9.quad9.net' fi # Obtain timeout if ! disable_error=1 G_CHECK_VALIDINT "$G_CHECK_URL_TIMEOUT" 0; then G_CHECK_URL_TIMEOUT=$(sed -n '/^[[:blank:]]*CONFIG_G_CHECK_URL_TIMEOUT=/{s/^[^=]*=//p;q}' /boot/dietpi.txt) disable_error=1 G_CHECK_VALIDINT "$G_CHECK_URL_TIMEOUT" 0 || G_CHECK_URL_TIMEOUT=10 fi # Obtain attempts if ! disable_error=1 G_CHECK_VALIDINT "$G_CHECK_URL_ATTEMPTS" 1; then G_CHECK_URL_ATTEMPTS=$(sed -n '/^[[:blank:]]*CONFIG_G_CHECK_URL_ATTEMPTS=/{s/^[^=]*=//p;q}' /boot/dietpi.txt) disable_error=1 G_CHECK_VALIDINT "$G_CHECK_URL_ATTEMPTS" 1 || G_CHECK_URL_ATTEMPTS=2 fi G_EXEC_RETRIES=$(( $G_CHECK_URL_ATTEMPTS - 1 )) # 2 attempts = 1 retry # Re-assign timeout ($5 => acommand[4]) on every retry, in case it is changed as solution attempt declare -F G_EXEC_PRE_FUNC &> /dev/null && eval "G_EXEC_PRE_FUNC_ORIG()$(declare -f G_EXEC_PRE_FUNC | tail -n +2)" G_EXEC_PRE_FUNC(){ acommand[4]=$G_CHECK_URL_TIMEOUT; declare -F G_EXEC_PRE_FUNC_ORIG &> /dev/null && G_EXEC_PRE_FUNC_ORIG; } G_EXEC_DESC='Checking DNS resolver' G_EXEC ping -nc 1 -W $G_CHECK_URL_TIMEOUT ${domain:-"$@"} local exit_code=$? unset -v G_CHECK_URL_TIMEOUT G_CHECK_URL_ATTEMPTS unset -f G_EXEC_PRE_FUNC_ORIG return $exit_code } # URL connection test # - Checks a specific HTTP/HTTPS/FTP online resource via its URL # - Required arguments: # $* = URL + optional curl arguments # - Optional variables: # G_CHECK_URL_TIMEOUT to override default and dietpi.txt set timeout # G_CHECK_URL_ATTEMPTS to override default and dietpi.txt set attempts G_CHECK_URL(){ # Obtain timeout if ! disable_error=1 G_CHECK_VALIDINT "$G_CHECK_URL_TIMEOUT" 0; then G_CHECK_URL_TIMEOUT=$(sed -n '/^[[:blank:]]*CONFIG_G_CHECK_URL_TIMEOUT=/{s/^[^=]*=//p;q}' /boot/dietpi.txt) disable_error=1 G_CHECK_VALIDINT "$G_CHECK_URL_TIMEOUT" 0 || G_CHECK_URL_TIMEOUT=10 fi # Obtain attempts if ! disable_error=1 G_CHECK_VALIDINT "$G_CHECK_URL_ATTEMPTS" 1; then G_CHECK_URL_ATTEMPTS=$(sed -n '/^[[:blank:]]*CONFIG_G_CHECK_URL_ATTEMPTS=/{s/^[^=]*=//p;q}' /boot/dietpi.txt) disable_error=1 G_CHECK_VALIDINT "$G_CHECK_URL_ATTEMPTS" 1 || G_CHECK_URL_ATTEMPTS=2 fi G_EXEC_RETRIES=$(( $G_CHECK_URL_ATTEMPTS - 1 )) # 2 attempts = 1 retry # Re-assign timeout ($3 => acommand[2]) on every retry, in case it is changed as solution attempt declare -F G_EXEC_PRE_FUNC &> /dev/null && eval "G_EXEC_PRE_FUNC_ORIG()$(declare -f G_EXEC_PRE_FUNC | tail -n +2)" G_EXEC_PRE_FUNC(){ acommand[2]=$G_CHECK_URL_TIMEOUT; declare -F G_EXEC_PRE_FUNC_ORIG &> /dev/null && G_EXEC_PRE_FUNC_ORIG; } # "--retry" only applies on "a timeout, an FTP 4xx response code or an HTTP 5xx response code", hence we loop ourself. G_EXEC_DESC="Checking URL: $*" G_EXEC curl -ILfvm $G_CHECK_URL_TIMEOUT "$@" local exit_code=$? unset -v G_CHECK_URL_TIMEOUT G_CHECK_URL_ATTEMPTS unset -f G_EXEC_PRE_FUNC_ORIG return $exit_code } #----------------------------------------------------------------------------------- # Print network details #----------------------------------------------------------------------------------- # Commands: # "gateway": Print the default gateway IP # "iface": Print the interface name # "ip": Print the IP address # Options: # "-q": Hide all error messages that are not related to invalid arguments # "-4": Print info for interfaces with an IPv4 address only if available, else return 1 # "-6": Print info for interfaces with an IPv6 address only if available, else return 1 # "-t TYPE": Print info for interfaces of type TYPE only if available, else return 1 # TYPE can be one of "eth" and "wlan". # "-i IFACE": Print info for the network interface named IFACE only it present, else return 1 # Notes: # Info is shown for the one matching interface, following the following priorities: # - 1. the interface which has the default gateway assigned # - 2. the first interface with state "UP" # - 3. the first interface with an IP address assigned # - 4. the first available interface # - If no interface exists, the function returns error 1. # If not defined, IPv4 addresses are shown if available, else IPv6 addresses if available. G_GET_NET() { # Grab input local quite=0 fam type iface command while (( $# )) do case "$1" in '-q') quite=1;; '-'[46]) fam=$1;; '-t') shift; type=$1;; '-i') shift; iface=$1;; 'gateway'|'iface'|'ip') command=$1;; *) G_DIETPI-NOTIFY 1 "An invalid argument \"${1:-<empty>}\" was given."; return 1;; esac shift done # A command is required [[ $command ]] || { G_DIETPI-NOTIFY 1 "No command was given."; return 1; } # Early return if given interface does not exists or does not match given type if [[ $iface ]] then if [[ ! -e /sys/class/net/$iface ]] then (( $quite )) || G_DIETPI-NOTIFY 1 "The given interface \"$iface\" does not exist." return 1 elif [[ $type && $iface != $type* ]] then (( $quite )) || G_DIETPI-NOTIFY 1 "The given interface \"$iface\" is not of type \"$type\"." return 1 fi fi # Get default gateway if requested or no interface given local ip if if [[ $command == 'gateway' || ! $iface ]] then local gateway [[ $fam != '-6' ]] && read -r gateway if < <(ip r l 0/0 ${iface:+dev "$iface"} | mawk '{print $3,$5;exit}') [[ ! $gateway && $fam != '-4' ]] && read -r gateway if < <(ip -6 r l ::/0 ${iface:+dev "$iface"} | mawk '{print $3,$5;exit}') # ip r does not print the interface name if one was given, so used the given one to check for type. [[ $iface ]] && if=$iface # Print default gateway if requested if [[ $command == 'gateway' ]] then if [[ $gateway ]] then # Check for interface type if [[ $type && $if != $type* ]] then (( $quite )) || G_DIETPI-NOTIFY 1 "The default gateway is not assigned to any interface of type \"$type\"." return 1 fi echo "$gateway" else (( $quite )) || G_DIETPI-NOTIFY 1 "A default gateway${fam:+ for IPv${fam#-}}${iface:+ on interface \"$iface\"} does not exist." return 1 fi return 0 fi # Print interface/IP address if matching default gateway was found if [[ $gateway && ( ! $type || $if == $type* ) ]] then iface=$if # shellcheck disable=SC2086 [[ $command == 'ip' ]] && ip=$(ip -br $fam a s dev "$iface" | mawk '{print $3;exit}') ip=${ip%/*} echo "${!command}" return 0 fi fi local state if_final ip_final # shellcheck disable=SC2086 while read -r if state ip do [[ $if == 'lo' ]] && continue [[ $type && $if != $type* ]] && continue # Cut off secondary IP addresses and CIDR mask ip=${ip%%/*} # Return state UP if [[ $state == 'UP' ]] then iface=$if echo "${!command}" return 0 fi # Store info in separate variables to return if no UP state interface was found if [[ $ip && ! $ip_final ]] then if_final=$if ip_final=$ip elif [[ ! $if_final ]] then if_final=$if fi done < <(ip -br $fam a ${iface:+s dev "$iface"}) # Return final values iface=$if_final ip=$ip_final if [[ $command == 'ip' && ! $ip ]] then (( $quite )) || G_DIETPI-NOTIFY 1 "An interface${iface:+ named \"$iface\"}${type:+ of type \"$type\"} with an IP${fam:+v${fam#-}} address does not exist." return 1 elif [[ $command == 'iface' && ! $iface ]] then (( $quite )) || G_DIETPI-NOTIFY 1 "An interface${iface:+ named \"$iface\"}${type:+ of type \"$type\"}${fam:+ with an IPv${fam#-} address} does not exist." return 1 fi echo "${!command}" return 0 } # Print public IP address and location info # - Optional arguments: # -t <timeout>: Set timeout in seconds, supports floats, default: 3 G_GET_WAN_IP() { # Defaults local timeout=3 # Inputs while (( $# )) do # shellcheck disable=SC2015 case $1 in '-t') shift; (( ${1/.} )) && timeout=$1 || { G_DIETPI-NOTIFY 1 "Invalid timeout \"$1\", aborting..."; return 1; };; *) G_DIETPI-NOTIFY 1 "Invalid argument \"$1\", aborting..."; return 1;; esac shift done curl -sSfLm "$timeout" 'https://dietpi.com/geoip' } # $1 = directory to test permissions support # Returns 0=ok >=1=failed G_CHECK_FS_PERMISSION_SUPPORT(){ local input=$1 exit_code=1 while : do if ! mkdir -p "$input"; then G_WHIP_MSG "Error creating directory $input, unable to check filesystem permissions" break fi local fp_target="$input/.test" if ! > "$fp_target"; then G_WHIP_MSG "Error creating test file $fp_target, unable to check filesystem permissions" break fi # Apply and check permissions support, twice (just in case the current value is already set) local permissions_failed=0 chmod 600 "$fp_target" if [[ $(stat -c "%a" "$fp_target") != '600' ]]; then permissions_failed=1 else chmod 644 "$fp_target" [[ $(stat -c "%a" "$fp_target") != '644' ]] && permissions_failed=1 fi if (( $permissions_failed )); then G_WHIP_MSG "ERROR: Filesystem does not support permissions (e.g.: FAT16/32):\n\n$fp_target\n\nPlease select a different drive and/or format it with ext4, ensuring support for filesystem permissions.\n\nUnable to continue, aborting..." break fi # Else ok exit_code=0 break done [[ -f $fp_target ]] && rm -f "$fp_target" return $exit_code } #----------------------------------------------------------------------------------- # APT: Non-interactive and error-handled wrappers for apt-get commands #----------------------------------------------------------------------------------- # Check for missing kernel modules, e.g. after a kernel upgrade, and in case create a flag G_CHECK_KERNEL() { [[ -d /lib/modules/$(uname -r) ]] || systemd-detect-virt -c > /dev/null || return 1 return 0 } # apt-get install G_AGI(){ # Return if no argument given (( $# )) || { G_DIETPI-NOTIFY 2 'No input package given. Aborting...'; return 0; } G_CHECK_ROOT_USER 1 G_EXEC_DESC="\e[0mAPT install for: \e[33m$*\e[0m" DEBIAN_FRONTEND=noninteractive G_EXEC_OUTPUT=1 G_EXEC_OUTPUT_COL='\e[90m' G_EXEC apt-get -qq --allow-change-held-packages install "$@" return $? } # apt-get purge G_AGP(){ # Return if no argument given (( $# )) || { G_DIETPI-NOTIFY 2 'No input package given. Aborting...'; return 0; } G_CHECK_ROOT_USER 1 # Attempt to purge only installed packages, check on every G_EXEC loop, force succeed if none were found, to avoid related apt-get error declare -F G_EXEC_PRE_FUNC &> /dev/null && eval "G_EXEC_PRE_FUNC_ORIG()$(declare -f G_EXEC_PRE_FUNC | tail -n +2)" G_EXEC_PRE_FUNC(){ local apackages=() mapfile -t apackages < <(dpkg --get-selections "${acommand[@]:4}" 2> /dev/null | mawk '{print $1}') # shellcheck disable=SC2015 [[ ${apackages[0]} ]] && acommand=("${acommand[@]::4}" "${apackages[@]}") || acommand=(G_DIETPI-NOTIFY 2 'None of the requested packages are currently installed. Aborting...') declare -F G_EXEC_PRE_FUNC_ORIG &> /dev/null && G_EXEC_PRE_FUNC_ORIG } G_EXEC_DESC="\e[0mAPT purge for: \e[33m$*\e[0m" DEBIAN_FRONTEND=noninteractive G_EXEC_OUTPUT=1 G_EXEC_OUTPUT_COL='\e[90m' G_EXEC apt-get -qq --allow-change-held-packages autopurge "$@" local exit_code=$? unset -f G_EXEC_PRE_FUNC_ORIG return $exit_code } # apt-get autopurge G_AGA(){ G_CHECK_ROOT_USER 1 G_EXEC_DESC='\e[0mAPT autopurge' DEBIAN_FRONTEND=noninteractive G_EXEC_OUTPUT=1 G_EXEC_OUTPUT_COL='\e[90m' G_EXEC apt-get -qq autopurge return $? } # apt-get -f install G_AGF(){ G_CHECK_ROOT_USER 1 G_EXEC_DESC='\e[0mAPT fix' DEBIAN_FRONTEND=noninteractive G_EXEC_OUTPUT=1 G_EXEC_OUTPUT_COL='\e[90m' G_EXEC apt-get -qq --allow-change-held-packages -f install return $? } # apt-get clean + update # - $1 = -f: Store number of available APT upgrades to file, which defaults to: /run/dietpi/.apt_updates # $2 = Optional file path to store number of available APT upgrades to # - $1 = -v: Store number of available APT upgrades to variable, which defaults to: G_AGUP_COUNT # $2 = Optional variable name to store number of available APT upgrades to # shellcheck disable=SC2120 G_AGUP(){ G_CHECK_ROOT_USER 1 # Clean cache before every update, which can corrupt and gets fully rewritten anyway declare -F G_EXEC_PRE_FUNC &> /dev/null && eval "G_EXEC_PRE_FUNC_ORIG()$(declare -f G_EXEC_PRE_FUNC | tail -n +2)" G_EXEC_PRE_FUNC(){ apt-get clean; declare -F G_EXEC_PRE_FUNC_ORIG &> /dev/null && G_EXEC_PRE_FUNC_ORIG; } # Fail when some index files couldn't be downloaded, e.g. due to DNS failure. Currently apt-get update prints a warning but does not return an error code. declare -F G_EXEC_POST_FUNC &> /dev/null && eval "G_EXEC_POST_FUNC_ORIG()$(declare -f G_EXEC_POST_FUNC | tail -n +2)" G_EXEC_POST_FUNC(){ [[ $exit_code == 0 && $(<$fp_log) == *'W: Some index files failed to download.'* ]] && exit_code=255; declare -F G_EXEC_POST_FUNC_ORIG &> /dev/null && G_EXEC_POST_FUNC_ORIG; } G_EXEC_DESC='\e[0mAPT update' G_EXEC_OUTPUT=1 G_EXEC_OUTPUT_COL='\e[90m' G_EXEC apt-get -q update local exit_code=$? unset -f G_EXEC_{PRE,POST}_FUNC_ORIG if [[ $1 == '-'[fv] ]] then local count=0 (( $exit_code )) || count=$(apt -qq list --upgradeable 2> /dev/null | wc -l) # Mute "apt" CLI warning if [[ $1 == '-f' ]] then local file=${2:-/run/dietpi/.apt_updates} if (( $count )) then G_DIETPI-NOTIFY 2 "Storing number of available APT upgrades to file: $file" echo "$count" > "$file" else G_DIETPI-NOTIFY 2 "No APT upgrades were found, not creating file: $file" [[ -f $file ]] && rm "$file" fi elif [[ $1 == '-v' ]] then local var=${2:-G_AGUP_COUNT} if (( $count )) then G_DIETPI-NOTIFY 2 "Storing number of available APT upgrades to variable: $var" declare -g "$var=$count" else G_DIETPI-NOTIFY 2 "No APT upgrades were found, not creating variable: $var" unset -v "$var" fi fi fi return $exit_code } # apt-get upgrade G_AGUG(){ G_CHECK_ROOT_USER 1 G_EXEC_DESC='\e[0mAPT upgrade' DEBIAN_FRONTEND=noninteractive G_EXEC_OUTPUT=1 G_EXEC_OUTPUT_COL='\e[90m' G_EXEC apt-get -qq upgrade return $? } # apt-get dist-upgrade G_AGDUG(){ G_CHECK_ROOT_USER 1 G_EXEC_DESC='\e[0mAPT dist-upgrade' DEBIAN_FRONTEND=noninteractive G_EXEC_OUTPUT=1 G_EXEC_OUTPUT_COL='\e[90m' G_EXEC apt-get -qq --allow-change-held-packages dist-upgrade return $? } # Checks for required APT packages, installs if needed. # $@ = list of required packages # NB: automatically error handled (G_EXEC) G_AG_CHECK_INSTALL_PREREQ(){ # Return if no argument given (( $# )) || return 0 G_DIETPI-NOTIFY 2 "Checking for required APT packages: \e[33m$*" G_CHECK_ROOT_USER 1 local i apackages=() for i in "$@" do dpkg-query -s "$i" &> /dev/null && continue apackages+=("$i") done [[ ${apackages[0]} ]] || return 0 G_AGUP G_AGI "${apackages[@]}" exit_code=$? return $? } #----------------------------------------------------------------------------------- # MISC: Commands #----------------------------------------------------------------------------------- # Treesize # - $1 = Optional input directory, e.g.: G_TREESIZE /var/cache/apt G_TREESIZE(){ du -B 1 -d 1 ${1:+"$1"} | sort -nr | mawk ' BEGIN { split("bytes,KiB,MiB,GiB,TiB", unit, ",") } { u=1 while ($1>=1024) { $1=$1/1024; u+=1 } $1=sprintf("%.1f %s", $1, unit[u]) print $0 }' } # Returns current CPU temp 'C # - print_full_info=1 Optional input to print full colour text output and temp warnings G_OBTAIN_CPU_TEMP() { # Read CPU temp from file local temp # - Odroid N2/ASUS/Sparky: Requires special case as in other array this would break SBC temp readouts with 2 zones if [[ ( $G_HW_MODEL == 15 || $G_HW_MODEL == 52 || $G_HW_MODEL == 70 ) && -f '/sys/class/thermal/thermal_zone1/temp' ]] then temp=$(</sys/class/thermal/thermal_zone1/temp) # - Others else # Array to store possible locations for temp read local i afp_temperature=( '/sys/devices/platform/coretemp.*/hwmon/hwmon*/temp*_input' # Intel Mini PCs: https://github.com/MichaIng/DietPi/issues/3172, https://github.com/MichaIng/DietPi/issues/3412 '/sys/class/thermal/thermal_zone0/temp' '/sys/devices/platform/sunxi-i2c.0/i2c-0/0-0034/temp1_input' '/sys/class/hwmon/hwmon0/device/temp_label' '/sys/class/hwmon/hwmon0/temp2_input' '/sys/class/hwmon/hwmon0/temp1_input' # Odroid C1 Armbian legacy Linux 5.4.40: https://dietpi.com/phpbb/viewtopic.php?p=24860#p24860 '/sys/class/thermal/thermal_zone1/temp' # Roseapple Pi, probably OrangePi's: https://dietpi.com/phpbb/viewtopic.php?t=8677 ) # Coders NB: Do NOT quote the array to allow coretemp file paths glob expansion! # shellcheck disable=SC2068 for i in ${afp_temperature[@]} do [[ -f $i ]] || continue temp=$(<"$i") [[ $temp -gt 0 ]] && break # Trust only positive temperatures for now (strings are treated as "0") done fi # Format output # - Check for valid value: We must always return a value, due to VM lacking this feature + benchmark online if [[ $temp -lt 1 ]] then temp='N/A' else # 2/5 digit output? (( $temp >= 200 )) && temp=$(( $temp / 1000 )) if [[ $print_full_info == 1 ]] then local temp_f=$(( $temp * 9/5 + 32 )) if (( $temp >= 70 )) then temp="\e[1;31mWARNING: $temp °C / $temp_f °F : Reducing the life of your device\e[0m" elif (( $temp >= 60 )) then temp="\e[38;5;202m$temp °C / $temp_f °F \e[90m: Running hot, not recommended\e[0m" elif (( $temp >= 50 )) then temp="\e[1;33m$temp °C / $temp_f °F \e[90m: Running warm, but safe\e[0m" elif (( $temp >= 40 )) then temp="\e[1;32m$temp °C / $temp_f °F \e[90m: Optimal temperature\e[0m" elif (( $temp >= 30 )) then temp="\e[1;36m$temp °C / $temp_f °F \e[90m: Cool runnings\e[0m" else temp="\e[1;36m$temp °C / $temp_f °F \e[90m: Who put me in the freezer!\e[0m" fi fi fi echo -e "$temp" } # Returns current CPU usage in % G_OBTAIN_CPU_USAGE(){ local usage=0 # ps: inaccurate but fast while read -r line # Aside reading raw, -r removes leading and trailing white spaces each line do line=${line/.} # Remove decimal dot ((usage+=${line#0})) # Remove leading zero, if present, then sum up done < <(ps --no-headers -eo %cpu) # Single core usage in xy.z # ps returns single core usage, so we divide by core count usage=$(printf '%.1f' "$(($usage*10/$G_HW_CPU_CORES+1))e-2") # Divide by 10 to compensate decimal dot removal, re-add decimal dot via printf conversion but assure last digit is rounded correctly echo "$usage" } # Check available free space on path, against input value (MiB) # - Returns 0=Ok, 1=insufficient space available # If $2 is not used, returns available space in MiB | info_autoscale=1 # Scales MiB to GiB if required and prints unit # - $1 = path # - $2 = Optional, free space (MiB) # EG: if (( $(G_CHECK_FREESPACE /path 100) )); then G_CHECK_FREESPACE(){ local info_autoscale=${info_autoscale:-0} local return_value=1 local input_path=$1 local input_required_space=$2 local available_space=$(df -m --output=avail "$input_path" | mawk 'NR==2 {print $1}') if ! disable_error=1 G_CHECK_VALIDINT "$available_space"; then G_WHIP_MSG 'G_CHECK_FREESPACE: Invalid integer from df result' elif [[ ! $input_required_space ]]; then (( $info_autoscale )) && { (( $available_space > 9999 )) && available_space="$(( $available_space / 1024 )) GiB" || available_space+=' MiB'; } echo "$available_space" return_value=0 else (( $available_space > $input_required_space )) && return_value=0 G_DIETPI-NOTIFY $return_value "Free space check: path=$input_path | available=$available_space MiB | required=$input_required_space MiB" fi return $return_value } # G_CHECK_VALIDINT | Simple test to verify if a variable is a valid integer. # $1=input # $2=Optional Min value range # $3=Optional Max value range # disable_error=1 to disable notify/whiptail invalid value when received # 1=no | scripts killed automatically # 0=yes # Usage = if G_CHECK_VALIDINT input; then G_CHECK_VALIDINT(){ local input=$1 min=$2 max=$3 return_value=1 if [[ $input =~ ^-?[0-9]+$ ]]; then if [[ $min =~ ^-?[0-9]+$ ]]; then if (( $input >= $min )); then if [[ $max =~ ^-?[0-9]+$ ]]; then if (( $input <= $max )); then return_value=0 elif [[ $disable_error != 1 ]]; then G_WHIP_MSG "Input value \"$input\" is higher than allowed \"$max\". No changes applied." fi else return_value=0 fi elif [[ $disable_error != 1 ]]; then G_WHIP_MSG "Input value \"$input\" is lower than allowed \"$min\". No changes applied." fi else return_value=0 fi elif [[ $disable_error != 1 ]]; then G_WHIP_MSG "Invalid input value \"$input\". No changes applied." fi unset -v disable_error return $return_value } # Verifies the integrity of the DietPi userdata folder/symlink, based on where it should be physically. Basically, checks if user removed the USB drive with userdata on it. # NB: As this is considered a critical (if failed), current scripts will be exited automatically # 1=fail # 0=ok G_CHECK_USERDATA(){ local return_value=0 fp_actual='/mnt/dietpi_userdata' # Symlinked? if [[ -L '/mnt/dietpi_userdata' ]] then # Check physical location exists (is mounted) fp_actual=$(readlink -f /mnt/dietpi_userdata) [[ -d $fp_actual ]] || return_value=1 fi G_DIETPI-NOTIFY $return_value "DietPi-Userdata validation: $fp_actual" (( $return_value )) || return 0 G_WHIP_MSG "[FAILED] DietPi-Userdata validation\n\nDietPi was unable to verify the existence of the userdata directory ($fp_actual).\n\nPlease ensure all previous external drives are connected and functional, before trying again.\n\nUnable to continue, exiting." kill -INT $$ # kill all current scripts, excluding shell return $return_value } # Prompt user to create a backup before system changes. Exit existing scripts if failed. G_PROMPT_BACKUP(){ [[ $G_PROMPT_BACKUP_DISABLED == 1 ]] && return 0 G_WHIP_YESNO 'Would you like to create (or update) a "DietPi-Backup" of the system, before proceeding?\n\n"DietPi-Backup" creates a system restore point, which can be recovered if unexpected issues occur.\n\nFor more information on "DietPi-Backup", please use the link below:\n - https://dietpi.com/docs/dietpi_tools/#dietpi-backup-backuprestore' || return 0 /boot/dietpi/dietpi-backup 1 local exit_code=$? G_DIETPI-NOTIFY -1 $exit_code 'DietPi-Backup' (( $exit_code )) || return 0 # Kill current scripts, excluding shell G_WHIP_MSG '[FAILED] DietPi-Backup was unable to complete sucessfully.\n\nTo avoid issues, the current program will now be terminated.\n\nLog file: /var/log/dietpi-backup.log' kill -INT $$ return 1 } # If file/folder exists, backup to *.bak_DDMMYYY G_BACKUP_FP(){ local ainput_string=("$@") local fp_db_log='/var/tmp/dietpi/logs/G_BACKUP_FP.db' local print_fp_db_info=0 local i for i in "${ainput_string[@]}" do [[ -e $i ]] || continue local fp_backup_target="$i.bak_$(date +%d%m%y)" local index=0 while [[ -e ${fp_backup_target}_$index ]] do ((index++)) done local notify_code=1 if cp -a "$i" "${fp_backup_target}_$index"; then notify_code=0 print_fp_db_info=1 echo "${fp_backup_target}_$index # $G_PROGRAM_NAME" >> $fp_db_log fi G_DIETPI-NOTIFY $notify_code "$i: backup to ${fp_backup_target}_$index" done (( $print_fp_db_info )) && G_DIETPI-NOTIFY 2 "For a full list of backup items, please see $fp_db_log" } # Apply and update to different branch G_DEV_BRANCH(){ G_CHECK_ROOT_USER 1 G_CONFIG_INJECT 'DEV_GITBRANCH=' "DEV_GITBRANCH=$1" /boot/dietpi.txt /boot/dietpi/dietpi-update -1 } # Inject setting into config file: First tries to replace old setting, else commented setting and otherwise adds to end of file. # Usage: # - $1 Setting pattern to find existing setting with grep extended regular expression support # - $2 Target setting + value, to inject into config file: After bash string expansion (e.g. variables), everything else will be taken literally, thus no further escaping is required. # - $3 Path to config file # - $4 (optional) Line pattern after which the setting will be added instead of end of file with grep extended regular expression support # - GCI_PASSWORD=1 G_CONFIG_INJECT, password entry, do not print raw output to screen. # - GCI_PRESERVE=1 G_CONFIG_INJECT preserves current setting, if present. # - GCI_BACKUP=1 G_CONFIG_INJECT creates a backup before editing the file, if backup does not yet exist, to: $3.bak # - GCI_NEWLINE=1 G_CONFIG_INJECT explicitly expands newlines \n within $2, which by default are taken literally # NB: Be careful with this, since pattern matching is only done per line which can lead to doubled lines when applying G_CONFIG_INJECT a second time. # NB: # - Within double quotes "", as usual, escape literally meant double quotes and dollar signs $ with leading backslash \. # - Within single quotes '', as usual, escape literally meant single quotes via: '\'' # End leading string; Add escaped single quote; Start trailing string # - Additionally in case of extended regular expression support ($1 and $4), the following characters need to be escaped via backslash \, if wanted literally: # \ . + * ? [ ( { ^ & $ | # Example: # - G_CONFIG_INJECT 'prefer-family[[:blank:]=]' 'prefer-family = IPv4' /etc/wgetrc G_CONFIG_INJECT(){ [[ $G_PROGRAM_NAME ]] || local G_PROGRAM_NAME='G_CONFIG_INJECT' local pattern=${1//\//\\\/} local setting_raw=$2 local setting=${2//\\/\\\\}; setting=${setting//./\\.}; setting=${setting//+/\\+}; setting=${setting//\*/\\\*}; setting=${setting//\?/\\\?}; setting=${setting//[/\\[} setting=${setting//\(/\\\(}; setting=${setting//\{/\\\{}; setting=${setting//^/\\^}; setting=${setting//&/\\&}; setting=${setting//$/\\$}; setting=${setting//|/\\|}; setting=${setting//\//\\\/} [[ $GCI_NEWLINE == 1 ]] && setting=${setting//\\\\n/\\n} local file=$3 local after=${4//\//\\\/} local error # Colouring output local yellow reset [[ -t 0 || -t 1 ]] && yellow='\e[33m' reset='\e[0m' # Replace password string by asterisks in output string if [[ $GCI_PASSWORD == 1 ]]; then local password=$(sed -E "s/^.*${pattern}[[:blank:]]*//" <<< "$setting_raw") setting_raw="$(sed -E "s/(^.*${pattern}[[:blank:]]*).*$/\1/" <<< "$setting_raw")${password//?/*}" unset -v password fi syntax_error(){ [[ $after ]] && after="after line \$4\n $after (raw escaped input)\n" [[ $error ]] && error="\n\"grep\" or \"sed\" reported the following error:\n $error\n" G_WHIP_MSG "[FAILED] Syntax error $error Couldn't add setting \$2 $setting (escaped input) into file \$3 $file $after NB: - Within double quotes \"\", as usual, escape literally meant double quotes and dollar signs \$ via: \\\" respectively \\\$ - Within single quotes '', as usual, escape literally meant single quotes via: '\'' # <End leading string>; <Add escaped single quote>; <Start trailing string> - Additionally in case of extended regular expression support (\$1 and \$4), the following characters need to be escaped via backslash \, if wanted literally: \ . + * ? [ ( { ^ & $ | - Do not escape forward slashes /, which will be done internally for all arguments!" unset -v syntax_error } if [[ ! -w $file ]]; then G_WHIP_MSG "[FAILED] File does not exist or cannot be written to by current user \nPlease verify the existence of the file \$3 $file \nRetry with proper permissions or apply the setting manually: $setting_raw" elif error=$(grep -Eq "^[[:blank:]]*$pattern" "$file" 2>&1); then # As an error within the condition leads to result "false", it can be caught only in next "elif"/"else" statement. if [[ $GCI_PRESERVE == 1 ]]; then # shellcheck disable=SC2015 G_DIETPI-NOTIFY 0 "Current setting in $yellow$file$reset will be preserved: $yellow$([[ $GCI_PASSWORD == 1 ]] && echo "${setting_raw//\\/\\\\}" || grep -Em1 "^[[:blank:]]*$pattern" "$file" | sed 's|\\|\\\\|g')$reset" elif error=$(grep -Eq "^[[:blank:]]*$setting([[:space:]]|$)" "$file" 2>&1); then # shellcheck disable=SC2015 G_DIETPI-NOTIFY 0 "Desired setting in $yellow$file$reset was already set: $yellow$([[ $GCI_PASSWORD == 1 ]] && echo "${setting_raw//\\/\\\\}" || grep -Em1 "^[[:blank:]]*$pattern" "$file" | sed 's|\\|\\\\|g')$reset" elif error=$( (( $(grep -Ec "^[[:blank:]]*$pattern" "$file" 2>&1) > 1 )) 2>&1); then [[ $error ]] && { syntax_error; return 1; } G_WHIP_MSG "[FAILED] Setting was found multiple times \nThe pattern \$1 $(sed -E "c\\$pattern" <<< '') was found multiple times in file \$3 $file \n____________ $(grep -En "^[[:blank:]]*$pattern" "$file") ____________ \nEither the pattern \$1 needs to be more specific or the desired setting can appear multiple times by design and it cannot be predicted which instance to edit. Please retry with more specific parameter \$1 or apply the setting manually: $setting_raw" else [[ $error ]] && { syntax_error; return 1; } [[ $GCI_BACKUP == 1 && ! -f $file.bak ]] && cp -a "$file" "$file.bak" && G_DIETPI-NOTIFY 2 "Config file backup created: $yellow$file.bak$reset" error=$(sed -Ei "0,/^[[:blank:]]*$pattern.*$/s//$setting/" "$file" 2>&1) || { syntax_error; return 1; } G_DIETPI-NOTIFY 0 "Setting in $yellow$file$reset adjusted: $yellow${setting_raw//\\/\\\\}$reset" fi elif error=$(grep -Eq "^[[:blank:]#;]*$pattern" "$file" 2>&1); then [[ $error ]] && { syntax_error; return 1; } [[ $GCI_BACKUP == 1 && ! -f $file.bak ]] && cp -a "$file" "$file.bak" && G_DIETPI-NOTIFY 2 "Config file backup created: $yellow$file.bak$reset" error=$(sed -Ei "0,/^[[:blank:]#;]*$pattern.*$/s//$setting/" "$file" 2>&1) || { syntax_error; return 1; } G_DIETPI-NOTIFY 0 "Comment in $yellow$file$reset converted to setting: $yellow${setting_raw//\\/\\\\}$reset" else [[ $error ]] && { syntax_error; return 1; } if [[ $after ]]; then if error=$(grep -Eq "^[[:blank:]]*$after" "$file" 2>&1); then [[ $GCI_BACKUP == 1 && ! -f $file.bak ]] && cp -a "$file" "$file.bak" && G_DIETPI-NOTIFY 2 "Config file backup created: $yellow$file.bak$reset" error=$(sed -Ei "0,/^[[:blank:]]*$after.*$/s//&\n$setting/" "$file" 2>&1) || { syntax_error; return 1; } G_DIETPI-NOTIFY 0 "Added setting $yellow${setting_raw//\\/\\\\}$reset to $yellow$file$reset after line $yellow$(grep -Em1 "^[[:blank:]]*$after" "$file" | sed 's|\\|\\\\|g')$reset" else [[ $error ]] && { syntax_error; return 1; } G_WHIP_MSG "[FAILED] Setting could not be added after desired line \nThe pattern \$4 $(sed -E "c\\$after" <<< '') could not be found in file \$3 $file \nPlease retry with valid parameter \$4 or apply the setting manually: $setting_raw" fi else [[ $GCI_BACKUP == 1 && ! -f $file.bak ]] && cp -a "$file" "$file.bak" && G_DIETPI-NOTIFY 2 "Config file backup created: $yellow$file.bak$reset" # The following sed does not work on empty files: [[ ! -s $file ]] && echo '# Added by DietPi:' >> "$file" error=$(sed -Ei "\$a\\$setting" "$file" 2>&1) || { syntax_error; return 1; } G_DIETPI-NOTIFY 0 "Added setting $yellow${setting_raw//\\/\\\\}$reset to end of file $yellow$file$reset" fi fi } #----------------------------------------------------------------------------------- : # Return exit code 0, by triggering null as last command to output #----------------------------------------------------------------------------------- } So it's not a piece of cake ! 🤯 Edited April 1, 2022 by TRS-80 put long output inside spoiler 0 Quote Link to comment Share on other sites More sharing options...
Werner Posted April 1, 2022 Share Posted April 1, 2022 Merged three posts. You may want to use spoiler ( ) for longer posts to reduce overall topic length. 1 Quote Link to comment Share on other sites More sharing options...
TRS-80 Posted April 1, 2022 Share Posted April 1, 2022 Yeah, I went ahead and put it into a spoiler (to fold away) but something that big (especially that likely exists publicly in some forge) I would have probably linked to, personally. Well, I give you credit for at least doing some investigation. Other than noticing the great length of that script, I will admit to not investing any time whatsoever studying it any further. However, my immediate thought is: Do you really want to rely on something so complex to continue to be maintained by a third party? Or to fork and maintain yourself? Especially given that bash scripting is admittedly not your cup of tea? Prior to seeing this script, I was already a bit skeptical about how far DietPi strays away from standard Debian ways of doing things. But that was based on very brief experience playing around with it only once some time ago. If we take a step back to the fundamental problem that is trying to be solved here, that seems to me essentially to be backup. And backup is good! However perhaps there is some other better way to go about that? 0 Quote Link to comment Share on other sites More sharing options...
Myron Posted April 4, 2022 Share Posted April 4, 2022 For backups I occasionally create a direct sector-by-sector image using Win32DiskImager. Usually when I've made a fair few user-land alterations, which reminds me that I am due to take another back-up soon. 0 Quote Link to comment Share on other sites More sharing options...
Sigma7 Posted April 9, 2022 Share Posted April 9, 2022 I need something like TimeShift or Systemback (other examples). I tested these two apps that I've always used on Linux distros, but they didn't work on Armbian. I'd like incremental snapshots of file system to be restored at a later date to undo all changes to the system. 0 Quote Link to comment Share on other sites More sharing options...
lanefu Posted April 9, 2022 Share Posted April 9, 2022 You're probably going to need to be a little more hands on with the tool you choose. I like borg and borgmatic. 2 Quote Link to comment Share on other sites More sharing options...
Sigma7 Posted April 9, 2022 Share Posted April 9, 2022 Hey guys, I searched for "snap" in the Software application and "Back In Time" appeared with a graphical interface 🤣 I'm so glad Armbian read my mind. 🤭 Now I'm trying to configure: I don't know which system folder I should choose for backup (in the future I want to restore only the system) and where to save the snapshots. 0 Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.