How to Set Up a Dedicated File Server for Storing User Uploads

Storing user-uploaded files efficiently is a crucial aspect of backend development. Instead of relying on third-party cloud storage, setting up a dedicated file server gives you full control over data security, scalability, and cost efficiency. In this tutorial, we’ll go through the steps to set up a file server on a Linux VPS and connect it to a Node.js backend running on a separate server for seamless file uploads.

Prerequisites

Before you start, ensure you have:

• A VPS or dedicated server for storing files (Ubuntu 22.04 recommended)
• A separate VPS or server running Node.js and Express
• Root or sudo access
• Basic knowledge of SSH and Linux commands

Step 1: Update Your File Server

First, update your system’s package list to ensure all dependencies are up to date:

sudo apt update && sudo apt upgrade -y

Step 2: Set Up the File Server

Create a Dedicated Storage Directory

Create a directory where user uploads will be stored:

sudo mkdir -p /var/www/uploads
sudo chown -R www-data:www-data /var/www/uploads
sudo chmod -R 755 /var/www/uploads

Set Up Nginx to Serve Files

Create a new Nginx configuration file:

sudo nano /etc/nginx/sites-available/fileserver

Add the following configuration:

server {
    listen 80;
    server_name files.yourdomain.com;
location /uploads/ {
    root /var/www/;
    autoindex on;
}

}

Enable the configuration:

sudo ln -s /etc/nginx/sites-available/fileserver /etc/nginx/sites-enabled/
sudo systemctl restart nginx

Now, your file server can be accessed at http://files.yourdomain.com/uploads/.

Step 3: Secure Your File Server

• Enable HTTPS using Let’s Encrypt:

  sudo apt install certbot python3-certbot-nginx
  sudo certbot --nginx -d files.yourdomain.com
  

• Restrict File Types: Configure Nginx to prevent malicious file uploads.
• Set Up Backups: Use rsync or cron jobs to back up files to another server.

Step 4: Connect the Node.js App (Running on a Separate Server)

Your Node.js app will handle file uploads and send them to the file server.

1. Install Required Dependencies

SSH into your Node.js server and install the necessary packages:

npm install express multer axios form-data fs-extra

2. Set Up Multer for Handling File Uploads

Create an upload.js file:

const multer = require("multer");
const storage = multer.memoryStorage(); // Store files in memory before sending to the file server
const upload = multer({ storage: storage });
module.exports = upload;

3. Implement CRUD API for File Management

Modify your server.js or app.js file:

const express = require("express");
const axios = require("axios");
const FormData = require("form-data");
const upload = require("./upload");
const fs = require("fs-extra");
const app = express();
const port = 5000;
// Upload File
app.post("/upload", upload.single("file"), async (req, res) => {
if (!req.file) {
return res.status(400).json({ error: "No file uploaded" });
}
try {
const formData = new FormData();
formData.append("file", req.file.buffer, req.file.originalname);
await axios.post("http://files.yourdomain.com/upload", formData, {
headers: { ...formData.getHeaders() },
});
res.json({
message: "File uploaded successfully!",
fileUrl: http://files.yourdomain.com/uploads/${req.file.originalname},
});
} catch (error) {
res.status(500).json({ error: "Failed to upload file", details: error.message });
}
});
// List Files
app.get("/files", async (req, res) => {
try {
const response = await axios.get("http://files.yourdomain.com/uploads/");
res.json({ files: response.data });
} catch (error) {
res.status(500).json({ error: "Failed to retrieve files", details: error.message });
}
});
// Delete File
app.delete("/delete/:filename", async (req, res) => {
try {
const filename = req.params.filename;
await axios.delete(http://files.yourdomain.com/uploads/${filename});
res.json({ message: "File deleted successfully" });
} catch (error) {
res.status(500).json({ error: "Failed to delete file", details: error.message });
}
});
// Update (Replace) File
app.put("/update/:filename", upload.single("file"), async (req, res) => {
if (!req.file) {
return res.status(400).json({ error: "No file uploaded" });
}
try {
const filename = req.params.filename;
const formData = new FormData();
formData.append("file", req.file.buffer, filename);
await axios.put(http://files.yourdomain.com/uploads/${filename}, formData, {
headers: { ...formData.getHeaders() },
});
res.json({
message: "File updated successfully!",
fileUrl: http://files.yourdomain.com/uploads/${filename},
});
} catch (error) {
res.status(500).json({ error: "Failed to update file", details: error.message });
}
});
app.listen(port, () => {
console.log(Node.js server running on http://localhost:${port});
});

Conclusion

Now, your Node.js app (running on one server) can handle file uploads and send them to a dedicated file server for storage, while also supporting CRUD operations.