Loop over json files inside subdirectories

I have a folder with subfolders in it. I try to loop over each subfolder and import json files in mongo database:

#!/bin/bash
for dir in /data/Mydata/*/;
do ls -1 *.json | while read jsonfile; do mongoimport --db MyApp --collection logs --file $jsonfile --type json done;
done;

But this gives me an error:

ls: cannot access *.json: No such file or directory

If I am inside each subdirectory and do:

ls -1 *.json | while read jsonfile;
do mongoimport --db MyApp --collection logs --file $jsonfile --type json
done;

It works.

What am I doing wrong? I am quite new to Ubuntu. Best Regards

2 Answers

You haven't referenced the "dir" from your for line in your do line, so bash doesn't know where to look for the *.json files that are in "$dir"

You could simplify by just matching the files:

for jsonfile in /data/Mydata/*/*.json ;
do mongoimport --db MyApp --collection logs --file "$jsonfile" --type json
done;

And thanks @terdon for fixing this so you don't have to parse ls, which is a very bad idea - avoid that in future!

I think the findcommand should achieve your goal, the script would become:

#!/bin/bash
find /data/Mydata/ -maxdepth 2 -mindepth 2 -iname "*.json" -type f -exec mongoimport --db MyApp --collection logs --file "{}" --type json \;

This will only access direct subdirectories of /data/Mydata, due to the maxdepth and mindepth settings. "{}" gets expanded to the name of the file found each time, and the mongoimport command is run each time a file is found.

I've insured that the mongoimport command won't run on any directories that may be named something.json by using -type f which limits the find result to files, I imagine you wont have any directories named like this, but it's just a precaution