Disk is full and I cannot use SSH

Question

I am getting an error when trying SSH in my instance. He tells me to check the console serial output. As I understand it, there is no more space on my disk. What am I doing? I need SSH to clean up the space, but cannot SSH because there is no space!

Here is what I see:

Starting OpenBSD Secure Shell server: sshdopen: No space left on device Oct 14 13:18:13 instance-1 sshd[2771]: Server listening on 0.0.0.0 port 22. [?25l[?1c7[1GOct 14 13:18:13 instance-1 sshd[2771]: Server listening on :: port 22. [[32m ok [39;49m8[?25h[?0c. udhcpd: Disabled. Edit /etc/default/udhcpd to enable it. mktemp: failed to create file via template `/tmp/tmp.XXXXXXXXXX': No space left on device mktemp: failed to create file via template `/tmp/tmp.XXXXXXXXXX': No space left on device mktemp: failed to create file via template `/tmp/tmp.XXXXXXXXXX': No space left on device mktemp: failed to create file via template `/tmp/tmp.XXXXXXXXXX': No space left on device % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 2384 100 2384 0 0 1126k 0 --:--:-- --:--:-- --:--:-- 2328k Oct 14 13:18:13 instance-1 google: {"attributes":{"sshKeys":"ishener:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAFbCZDZDvuIxUbH5AHYeUU/WUWaOBYI1S7Yl9k3oVFwrenn6XsMdDHKiSH2VtpenQ7mHu3YcLDFe0pO1AwJjnSO39JR/3tTVLeVbuHDTEhOhDHt0NE84S1rqHX6r591IDwLhoGnFdNibGs0Sc0uyR/kRxl5hjAWdskOm4wzald+uRctBd+hbdBmt6az7iF2UzHEV362LxUtIzaYWoo1hnhld07+eimi6t+bUHsgqDkVGaEUUDaRFWTaNlFI9UW/AMYOcu9C24molfpPKwe2R5C5HLI+8nNI7qvoGtrUZww7K5bxNQPe+bMvVitArjYItYNDU6OXvQVA/u6gnqnbt1MM= google-ssh {\"userName\":\" [e-mail] \",\"expireOn\":\"2015-10-14T13:13:05+0000\"}\nishener:ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCKT2j4VbRM6DXSjLb5UlOdzfaB4K2TvZHWGV3JD7T++EbWX87JLOKg6AdbDPWTlgKRan02TIT/Xshy28r7fCCc= google-ssh {\"userName\":\" [e-mail] \",\"expireOn\":\"2015-10-14T13:12:58+0000\"}\nrsa-key-20150806:ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAQEAxtMUn200CaOuRa8bHFuqrjDiyUDrLECUf9V/ZpxT24lrqEbS1bDT7oWQwcuxQZEcrTnfeCEDeIwQpbNoOGp8NufrZUUG8jpVnVQqCHQZ3T+0Gs6et6JYxldhb0xT3KJVwQM+qnZOfwsk7co/+XOhE63u62NvUlqpGDQQUvuFY1wV3B7Rfjhg2JYEHCMswLRnSfnvyxp6+uQJ Oct 14 13:18:13 instance-1 google: 4THd/FlcGQJyJHUvpVQAqBIii8yc59+Rb32Xlyii6YU4+G60dfP6ON1iX2qkxJT5/mIkPfd3yPizbGsYhJbaqNQHPUE9hdqTlfk3gyA8S6SySNwViQtUqOH+sbo+suiJHHwr67V/qw== rsa-key-20150806\n"},"cpuPlatform":"Intel Ivy Bridge","description":"","disks":[{"deviceName":"instance-1","index":0,"mode":"READ_WRITE","type":"PERSISTENT"}],"hostname":"instance-1.c.united-wavelet-102819.internal","id":1871676137734806120,"image":"","machineType":"projects/273410245967/machineTypes/g1-small","maintenanceEvent":"NONE","networkInterfaces":[{"accessConfigs":[{"externalIp":"104.197.52.39","type":"ONE_TO_ONE_NAT"}],"forwardedIps":[],"ip":"10.240.238.207","network":"projects/273410245967/networks/default"}],"scheduling":{"automaticRestart":"TRUE","onHostMaintenance":"MIGRATE"},"serviceAccounts":{" [e-mail] ":{"aliases":["default"],"email":" [e-mail] ","scopes":["https://www.googleapis.com/auth/devstorage.read_only","https://www.googleapis.com/auth/logging.write"]},"default":{"al Oct 14 13:18:13 instance-1 google: iases":["default"],"email":" [e-mail] ","scopes":["https://www.googleapis.com/auth/devstorage.read_only","https://www.googleapis.com/auth/logging.write"]}},"tags":["http-server","https-server"],"virtualClock":{"driftToken":"12943060760861539723"},"zone":"projects/273410245967/zones/us-central1-f"} Oct 14 13:18:13 instance-1 google: No startup script found in metadata. [....] startpar: service(s) returned failure: tomcat7 ...[?25l[?1c7[1G[[31mFAIL[39;49m8[?25h[?0c [31mfailed![39;49m Oct 14 13:18:40 instance-1 accounts-from-metadata: WARNING Could not update /home/rsa-key-20150806/.ssh/authorized_keys due to [Errno 2] No usable temporary directory found in ['/tmp', '/var/tmp', '/usr/tmp', '/'] Oct 14 13:18:40 instance-1 accounts-from-metadata: WARNING Could not update /home/ishener/.ssh/authorized_keys due to [Errno 2] No usable temporary directory found in ['/tmp', '/var/tmp', '/usr/tmp', '/'] Oct 14 13:18:40 instance-1 accounts-from-metadata: WARNING Could not update /home/ishener_zaph/.ssh/authorized_keys due to [Errno 2] No usable temporary directory found in ['/tmp', '/var/tmp', '/usr/tmp', '/'] Oct 14 13:18:45 instance-1 sshd[2884]: Connection closed by 173.194.92.49 [preauth] Oct 14 13:18:46 instance-1 sshd[2886]: Connection closed by 173.194.92.49 [preauth] Oct 14 13:18:47 instance-1 accounts-from-metadata: WARNING Could not update /home/rsa-key-20150806/.ssh/authorized_keys due to [Errno 2] No usable temporary directory found in ['/tmp', '/var/tmp', '/usr/tmp', '/'] Oct 14 13:18:47 instance-1 accounts-from-metadata: WARNING Could not update /home/ishener/.ssh/authorized_keys due to [Errno 2] No usable temporary directory found in ['/tmp', '/var/tmp', '/usr/tmp', '/'] Oct 14 13:18:47 instance-1 accounts-from-metadata: WARNING Could not update /home/ishener_zaph/.ssh/authorized_keys due to [Errno 2] No usable temporary directory found in ['/tmp', '/var/tmp', '/usr/tmp', '/'] Oct 14 13:18:49 instance-1 sshd[2903]: Connection closed by 173.194.92.52 [preauth] Oct 14 13:18:51 instance-1 sshd[2905]: Connection closed by 173.194.92.52 [preauth] Oct 14 13:18:55 instance-1 sshd[2907]: Connection closed by 173.194.92.52 [preauth] Oct 14 13:19:02 instance-1 sshd[2909]: Connection closed by 173.194.92.48 [preauth] Oct 14 13:19:17 instance-1 sshd[2912]: Connection closed by 173.194.92.48 [preauth] Oct 14 13:19:58 instance-1 accounts-from-metadata: WARNING Could not update /home/ishener_zaph/.ssh/authorized_keys due to [Errno 2] No usable temporary directory found in ['/tmp', '/var/tmp', '/usr/tmp', '/'] Oct 14 13:19:58 instance-1 accounts-from-metadata: WARNING Could not update /home/rsa-key-20150806/.ssh/authorized_keys due to [Errno 2] No usable temporary directory found in ['/tmp', '/var/tmp', '/usr/tmp', '/'] Oct 14 13:19:58 instance-1 accounts-from-metadata: WARNING Could not update /home/ishener/.ssh/authorized_keys due to [Errno 2] No usable temporary directory found in ['/tmp', '/var/tmp', '/usr/tmp', '/'] Oct 14 13:20:01 instance-1 accounts-from-metadata: WARNING Could not update /home/ishener_zaph/.ssh/authorized_keys due to [Errno 2] No usable temporary directory found in ['/tmp', '/var/tmp', '/usr/tmp', '/'] Oct 14 13:20:01 instance-1 accounts-from-metadata: WARNING Could not update /home/rsa-key-20150806/.ssh/authorized_keys due to [Errno 2] No usable temporary directory found in ['/tmp', '/var/tmp', '/usr/tmp', '/'] Oct 14 13:20:01 instance-1 accounts-from-metadata: WARNING Could not update /home/ishener/.ssh/authorized_keys due to [Errno 2] No usable temporary directory found in ['/tmp', '/var/tmp', '/usr/tmp', '/'] Oct 14 13:20:02 instance-1 sshd[2929]: Connection closed by 173.194.92.51 [preauth] Oct 14 13:20:03 instance-1 sshd[2946]: Connection closed by 173.194.92.50 [preauth] Oct 14 13:20:06 instance-1 sshd[2948]: Connection closed by 173.194.92.49 [preauth] Oct 14 13:20:09 instance-1 sshd[2950]: Connection closed by 173.194.92.51 [preauth] Oct 14 13:20:13 instance-1 sshd[2952]: Connection closed by 173.194.92.50 [preauth] Oct 14 13:20:25 instance-1 sshd[2955]: Connection closed by 173.194.92.49 [preauth] Oct 14 13:20:36 instance-1 sshd[2957]: Connection closed by 173.194.92.52 [preauth] Oct 14 13:20:55 instance-1 sshd[2959]: Connection closed by 173.194.92.52 [preauth] Oct 14 13:21:01 instance-1 accounts-from-metadata: WARNING Could not update /home/ishener_zaph/.ssh/authorized_keys due to [Errno 2] No usable temporary directory found in ['/tmp', '/var/tmp', '/usr/tmp', '/'] Oct 14 13:21:01 instance-1 accounts-from-metadata: WARNING Could not update /home/rsa-key-20150806/.ssh/authorized_keys due to [Errno 2] No usable temporary directory found in ['/tmp', '/var/tmp', '/usr/tmp', '/'] Oct 14 13:21:01 instance-1 accounts-from-metadata: WARNING Could not update /home/ishener/.ssh/authorized_keys due to [Errno 2] No usable temporary directory found in ['/tmp', '/var/tmp', '/usr/tmp', '/'] Oct 14 13:21:34 instance-1 sshd[2977]: Connection closed by 173.194.92.51 [preauth] Oct 14 13:23:01 instance-1 accounts-from-metadata: WARNING Could not update /home/rsa-key-20150806/.ssh/authorized_keys due to [Errno 2] No usable temporary directory found in ['/tmp', '/var/tmp', '/usr/tmp', '/'] Oct 14 13:23:01 instance-1 accounts-from-metadata: WARNING Could not update /home/ishener_zaph/.ssh/authorized_keys due to [Errno 2] No usable temporary directory found in ['/tmp', '/var/tmp', '/usr/tmp', '/'] Oct 14 13:23:01 instance-1 accounts-from-metadata: WARNING Could not update /home/ishener/.ssh/authorized_keys due to [Errno 2] No usable temporary directory found in ['/tmp', '/var/tmp', '/usr/tmp', '/'] 

Answer 1

You have various options for resolving this issue:

• Check if your operating system supports automatic resizing : If so, using the Cloud Console, you can edit the root root disk and increase its size. A virtual machine instance can automatically resize the partition to recognize additional space after the instance is restarted.

• Use the interactive serial console feature to log into your virtual machine and erase your virtual disk or copy them to another storage if you need them later.

• If you know what data you want to delete, you can configure the startup script to delete the files and reboot your VM to run the script (for example, rm / tmp / *).

• You can detach a permanent drive and attach this drive to another computer as an additional drive. On a temporary machine, you can mount it and clear the data or copy it to another storage if you need it later. Finally, recreate the original instance with the same boot disk. You can follow the same steps described in this video to add your disk to another Linux virtual machine, but add an existing boot disk instead of creating a new disk.

• Check if your operating system supports automatic resizing : if so, create a snapshot of your permanent disk, create a new permanent disk with a larger size from the snapshot. Finally, recreate the original instance with this large boot disk.