Use Case: Call History Service

The goal of this project is to build the service that will sync new files periodically from the service provider, that keeps the files for the short period. Then extracts the records from the CSV files and keep the data for the future use to provide the call history for each customer via API.

This is a small service that builds on top of LFTP and Cron to sync the files from the provider FTP. Check the simple implemenatation for details.

I have run a simple Node.js script that builds on top of Chokidar library used to monitor the files changes and MongoDB to keep the records that extracted from the files.

This service provides the call history and communicate with the other services via the API. I have build API on top of hapi.js node framework.

upd: Sometime later I faced with a system limit on the number of files script can monitor. Listen uses inotify by default on Linux to monitor directories for changes.

For example, Ubuntu Lucid’s (64bit) inotify limit is set to 8192.

If you like to make your limit permanent, use:

echo fs.inotify.max_user_watches=524288 | sudo tee -a /etc/sysctl.conf
sudo sysctl -p

Check this thread for details.