Unable to start mongod… (no space left on device)

Question

Anyone know how to fix this? Exception is no space left on the device but all of my files only used 600mb/10GB.... I did restart mongod, --repair, uninstall and install but none of it works... Ple

2018-10-03T05:42:32.011+0000 I CONTROL [initandlisten] MongoDB starting : pid=2288 port=27017 dbpath=/data/db 64-bit host=itemtory 2018-10-03T05:42:32.011+0000 I CONTROL [initandlisten] db version v3.2.21 2018-10-03T05:42:32.011+0000 I CONTROL [initandlisten] git version: 1ab1010737145ba3761318508ff65ba74dfe8155 2018-10-03T05:42:32.011+0000 I CONTROL [initandlisten] OpenSSL version: OpenSSL 1.0.2g 1 Mar 2016 2018-10-03T05:42:32.012+0000 I CONTROL [initandlisten] allocator: tcmalloc 2018-10-03T05:42:32.012+0000 I CONTROL [initandlisten] modules: none 2018-10-03T05:42:32.012+0000 I CONTROL [initandlisten] build environment: 2018-10-03T05:42:32.012+0000 I CONTROL [initandlisten] distmod: ubuntu1604 2018-10-03T05:42:32.012+0000 I CONTROL [initandlisten] distarch: x86_64 2018-10-03T05:42:32.012+0000 I CONTROL [initandlisten] target_arch: x86_64 2018-10-03T05:42:32.012+0000 I CONTROL [initandlisten] options: {} 2018-10-03T05:42:32.037+0000 I - [initandlisten] Detected data files in /data/db created by the 'wiredTiger' storage engine, so setting the active storage engine to 'wiredTiger'. 2018-10-03T05:42:32.037+0000 W - [initandlisten] Detected unclean shutdown - /data/db/mongod.lock is not empty. 2018-10-03T05:42:32.037+0000 W STORAGE [initandlisten] Recovering data from the last clean checkpoint. 2018-10-03T05:42:32.038+0000 I STORAGE [initandlisten] wiredtiger_open config: create,cache_size=1G,session_max=20000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),checkpoint=(wait=60,log_size=2GB),statistics_log=(wait=0),verbose=(recovery_progress), 2018-10-03T05:42:32.135+0000 I STORAGE [initandlisten] WiredTiger [1538545352:135307][2288:0x7fbc2fb35c80], txn-recover: Main recovery loop: starting at 6/128 2018-10-03T05:42:32.135+0000 I STORAGE [initandlisten] WiredTiger [1538545352:135922][2288:0x7fbc2fb35c80], txn-recover: Recovering log 6 through 7 2018-10-03T05:42:32.137+0000 I STORAGE [initandlisten] WiredTiger [1538545352:137457][2288:0x7fbc2fb35c80], file:sizeStorer.wt, txn-recover: Recovering log 7 through 7 2018-10-03T05:42:32.216+0000 I CONTROL [initandlisten] ** WARNING: You are running this process as the root user, which is not recommended. 2018-10-03T05:42:32.216+0000 I CONTROL [initandlisten] 2018-10-03T05:42:32.217+0000 I CONTROL [initandlisten] 2018-10-03T05:42:32.217+0000 I CONTROL [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/defrag is 'always'. 2018-10-03T05:42:32.217+0000 I CONTROL [initandlisten] ** We suggest setting it to 'never' 2018-10-03T05:42:32.217+0000 I CONTROL [initandlisten] 2018-10-03T05:42:32.218+0000 I FTDC [initandlisten] Initializing full-time diagnostic data capture with directory '/data/db/diagnostic.data' 2018-10-03T05:42:32.218+0000 I NETWORK [initandlisten] waiting for connections on port 27017 2018-10-03T05:42:32.223+0000 I NETWORK [HostnameCanonicalizationWorker] Starting hostname canonicalization worker 2018-10-03T05:42:33.000+0000 W FTDC [ftdc] Uncaught exception in 'UnknownError: Caught std::exception of type boost::filesystem::filesystem_error: boost::filesystem::create_directory: No space left on device: "/data/db/diagnostic.data"' in full-time diagnostic data capture subsystem. Shutting down the full-time diagnostic data capture subsystem.

df -h:

Filesystem      Size  Used Avail Use% Mounted on
udev            991M     0  991M   0% /dev
tmpfs           201M   16M  185M   8% /run
/dev/vda1        25G    25G    0 100% /
tmpfs          1001M     0 1001M   0% /dev/shm
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs          1001M     0 1001M   0% /sys/fs/cgroup
/dev/vda15      105M  3.4M  102M   4% /boot/efi
tmpfs           201M     0  201M   0% /run/user/0

df -i /data/db:

Filesystem      Inodes  IUsed   IFree IUse% Mounted on
/dev/vda1      3225600 192164 3033436    6% /

Either it's some kind of insufficient permission situation, or you forgot to count some files. Run df -h, what does it say? — Sergio Tulentsev
Running out of inodes (although the last time something like that happened to me must have been 20-odd years ago)? df -i /data/db — robertklep
@Terence can you add the full output for both suggested df commands to your question? — robertklep

robertklep robertklep · Accepted Answer · 2018-10-04T06:08:29

The last command (df -i /data/db) shows that /data/db is located on the root (/) partition.

The first command (df -h) shows that that partition is 100% full. So that's the reason why you're getting a "No space left on device" error.

I'm not sure where you're getting "600mb/10GB" from, the root partition's size is 25GB.

Unable to start mongod… (no space left on device)

1 Answers