When working on Node.js applications, there are many times when you need to transfer files to remote servers. While FTP is still in use, SFTP (Secure File Transfer Protocol) is more secure and increasingly preferred. Today, I’ll walk you through how to upload a folder of files to an SFTP server using TypeScript and the ssh2-sftp-client library.
Thank me by sharing on Twitter 🙏
I’ll break this down into clear steps, from setting up the SFTP connection to recursively uploading files and ensuring the target directories exist. The process may seem a bit daunting at first, but by the end, you’ll have a working solution.
Why SFTP?
Before we get into the how-to, let’s briefly touch on why SFTP is a great choice. SFTP provides a secure way to transfer files over a network. Unlike FTP, which sends data in plain text, SFTP encrypts your data, making it much safer. This is especially important when dealing with sensitive information. Plus, SFTP is widely supported and relatively easy to set up in many hosting environments.
Installing ssh2-sftp-client
To begin, you’ll need to install the ssh2-sftp-client package, which is a well-maintained SFTP client library for Node.js. It handles the complexity of connecting and transferring files via SFTP with minimal configuration.
To install it, run:
Nexus: A Brief History of Information Networks from the Stone Age to AI
$21.66 (as of December 21, 2024 19:39 GMT +00:00 - More infoProduct prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on [relevant Amazon Site(s), as applicable] at the time of purchase will apply to the purchase of this product.)
Genesis: Artificial Intelligence, Hope, and the Human Spirit
$21.00 (as of December 21, 2024 19:39 GMT +00:00 - More infoProduct prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on [relevant Amazon Site(s), as applicable] at the time of purchase will apply to the purchase of this product.)
Co-Intelligence: Living and Working with AI
$13.78 (as of December 21, 2024 19:39 GMT +00:00 - More infoProduct prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on [relevant Amazon Site(s), as applicable] at the time of purchase will apply to the purchase of this product.)npm install ssh2-sftp-clientWe’ll also need a way to load environment variables securely. For that, we’ll use the dotenv package:
npm install dotenvThis allows us to keep sensitive information, like server credentials, out of the main code.
Setting Up Environment Variables
Instead of hardcoding sensitive information such as your server’s credentials directly in your script, it’s better to store them in an .env file. This way, you can load them dynamically and keep your code secure and flexible.
Here’s an example of what your .env file might look like:
SFTP_HOST=your-sftp-server.com
SFTP_PORT=22
SFTP_USER=your-username
SFTP_PASSWORD=your-passwordMake sure to replace these values with your actual SFTP server credentials. The SFTP_PORT defaults to 22, which is standard for SFTP, but you can change it if necessary.
Building the Upload Function
Now that we have the ssh2-sftp-client package installed and environment variables set up, let’s write the TypeScript code to upload a folder of files to the SFTP server. We want our solution to be flexible and handle different file types, subfolders, and directory creation if needed.
First, load your environment variables and import the required packages:
import * as dotenv from "dotenv";
import * as path from "path";
import * as fs from "fs-extra";
import SftpClient from "ssh2-sftp-client";
dotenv.config();Creating Directories on the Server
One thing you’ll encounter when working with SFTP is the need to ensure that the destination directory exists. If the directory doesn’t exist, your upload will fail. To address this, we need to create a helper function that checks if a directory exists on the server and, if not, creates it.
Here’s how you can handle that:
async function ensureRemoteDirExists(sftp: SftpClient, remoteFolder: string) {
const dirExists = await sftp.exists(remoteFolder);
if (!dirExists) {
console.log(`Creating remote folder: ${remoteFolder}`);
await sftp.mkdir(remoteFolder, true); // Recursive create
} else {
console.log(`Remote folder ${remoteFolder} already exists.`);
}
}This function checks for the existence of the directory using sftp.exists() and, if necessary, creates it using sftp.mkdir(). The true flag ensures that the directory is created recursively if needed.
Uploading Files and Folders Recursively
Next, let’s write the core function that uploads the contents of a local folder to the remote SFTP server. It needs to:
- Read all files and directories from the local folder.
- Ensure that remote directories exist.
- Upload each file, recursively processing subdirectories.
Here’s what that function looks like:
async function uploadFolderToSftp(localFolder: string, remoteFolder: string) {
const sftp = new SftpClient();
try {
// Connect to SFTP server using credentials from .env
await sftp.connect({
host: process.env.SFTP_HOST!,
port: parseInt(process.env.SFTP_PORT || "22", 10),
username: process.env.SFTP_USER!,
password: process.env.SFTP_PASSWORD!,
});
// Ensure the remote folder exists
await ensureRemoteDirExists(sftp, remoteFolder);
// Get the list of files in the local folder
const files = await fs.readdir(localFolder);
for (const file of files) {
const localFilePath = path.join(localFolder, file);
const remoteFilePath = path.join(remoteFolder, file);
const stats = await fs.stat(localFilePath);
if (stats.isDirectory()) {
// Recursively upload subfolders
await uploadFolderToSftp(localFilePath, remoteFilePath);
} else {
// Upload file
console.log(`Uploading ${localFilePath} to ${remoteFilePath}`);
await sftp.put(localFilePath, remoteFilePath);
}
}
console.log("All files uploaded successfully.");
} catch (err) {
console.error("Error uploading files:", err);
} finally {
// Close the SFTP connection
await sftp.end();
}
}Here’s what’s happening in this function:
- Connecting to the SFTP server: We use the credentials loaded from the
.envfile. - Ensure directories exist: For each file or folder, the script checks if the remote directory exists before uploading.
- Upload files: It uploads files one by one using
sftp.put(), and if it encounters a directory, it recursively calls the same function to handle subfolders.
Handling Permission Issues
One issue you might encounter is permission errors, especially when creating directories on the server. If you see an error like Permission denied, it’s likely that the SFTP user account doesn’t have sufficient privileges to write to the target directory.
Make sure that the user has write permissions in the target folder. In some cases, this might require changing the path to a directory where the user has access, like the user’s home directory.
Final Steps
Once you’ve written and tested your script, make sure you run it like any Node.js script. Use tsc to compile it to JavaScript if needed, then execute it. The script will handle creating directories, checking if files exist, and uploading everything, including nested subdirectories.
If you run into any errors related to permissions or folder existence, the script will log those errors, helping you debug and resolve issues.
Conclusion
SFTP is a secure and efficient way to transfer files, and with ssh2-sftp-client, it’s easy to automate the process using TypeScript. In this post, I showed how to upload an entire folder, recursively create directories, and handle common issues like missing folders or permission problems. By utilizing environment variables, we keep our credentials secure and flexible. I hope this guide helps you integrate SFTP functionality into your projects, making your file transfer processes secure and reliable.
