Downloading and saving files from the internet is a common programming task used in automation, data processing, and web scraping. This article explores how to download files programmatically using different tools and programming languages.
1. Using curl
(Command Line)
curl
is a versatile command-line tool for transferring data.
Example:
curl -o filename.txt https://example.com/file.txt
-o filename.txt
: Saves the file asfilename.txt
.https://example.com/file.txt
: URL of the file to download.
For files requiring authentication:
curl -u username:password -o filename.txt https://example.com/protected-file.txt
2. Using wget
(Command Line)
wget
is another command-line tool designed for downloading files.
Example:
wget -O filename.txt https://example.com/file.txt
-O filename.txt
: Saves the file with the specified name.
To download multiple files:
wget -i urls.txt
urls.txt
: A file containing a list of URLs to download.
3. Using Python
Python provides several libraries for downloading files.
3.1 Using requests
Install the requests
library if not already installed:
pip install requests
Example:
import requests
url = 'https://example.com/file.txt'
response = requests.get(url)
with open('file.txt', 'wb') as file:
file.write(response.content)
3.2 Using urllib
The urllib
module is part of Python’s standard library.
Example:
import urllib.request
url = 'https://example.com/file.txt'
urllib.request.urlretrieve(url, 'file.txt')
4. Using Java
Java’s HttpURLConnection
and NIO
libraries can handle file downloads.
Example:
import java.io.*;
import java.net.*;
public class FileDownloader {
public static void main(String[] args) {
String fileURL = "https://example.com/file.txt";
String saveDir = "file.txt";
try (BufferedInputStream in = new BufferedInputStream(new URL(fileURL).openStream());
FileOutputStream fileOutputStream = new FileOutputStream(saveDir)) {
byte[] dataBuffer = new byte[1024];
int bytesRead;
while1 != -1) {
fileOutputStream.write(dataBuffer, 0, bytesRead);
}
System.out.println("File downloaded successfully.");
} catch (IOException e) {
System.out.println("Error: " + e.getMessage());
}
}
}
5. Using C#
C# provides the WebClient
and HttpClient
classes for downloading files.
Example Using WebClient
:
using System.Net;
class Program
{
static void Main(string[] args)
{
string url = "https://example.com/file
- bytesRead = in.read(dataBuffer, 0, 1024 [↩]