A customer uploads a 500MB report through your application. The request hits your API, sits in memory while the server processes it, and then gets forwarded to S3. Now multiply that by 20 users doing the same thing at the same time. 

Your API layer starts to choke. Memory usage climbs, request queues build up, and response times degrade across unrelated endpoints. The system spends more time handling file transfer than executing business logic. 

A similar pattern shows up in any system that handles documents, images, or media at scale. The architecture looks straightforward, but the data path works against you. 

Pre-signed URLs change that path. They let the client upload directly to S3 while the backend controls access through short-lived permissions. 

In this blog, we will explore: 

1. The bottlenecks of traditional upload methods that slow down your backend 

2. The security risks of common shortcuts that expose your infrastructure 

3. A step-by-step breakdown of how pre-signed URLs solve these issues 

4. The practical trade-offs you need to know before moving to production 

The Efficiency Trap: Why Traditional Uploads Fail 

Applications that handle documents, images, or media often start with a simple upload design. The client sends the file to the backend, and the backend pushes it to S3. 

Flow: 
User → Backend → S3 

What happens in this setup 

• The backend receives the full file and holds the request open during upload.  

• The server processes and forwards the same file to S3.  

• The same data moves through your system twice, increasing network usage.  

• Large files occupy memory and worker threads for longer durations.  

What breaks under load 

• Concurrent uploads increase memory pressure and request queue time.  

• API latency rises across unrelated endpoints.  

• Backend resources shift toward file transfer instead of business logic.  

• Scaling the API layer increases cost without solving the core issue. 

Alternative Flow 
User → S3 (using frontend credentials) 

What improves 

• Files move directly to S3 without passing through the backend.  

• Upload speed increases due to a single network path. 

• Backend load drops immediately. 

What goes wrong 

• Frontend code exposes AWS credentials. 

• Anyone can extract keys from browser network requests.  

• Exposed credentials grant access based on assigned permissions.  

• A single leak can compromise the entire storage layer. 

A Better Path: How Pre-Signed URLs Work 

Pre-signed URLs solve both performance and security issues by letting the frontend communicate directly with S3, permissioned link generated by the backend. 

The Workflow 

1. The Request: The frontend asks the backend for an upload URL. 

2. The Permission: The backend generates a temporary, cryptographically signed URL. 

3. The Handover: The backend sends that URL back to the frontend. 

4. The Direct Lane: The frontend uploads the file directly to S3 using that URL. 

5. The Wrap-up: Once the upload is done, the backend saves the file’s metadata for future use. 

Feature	Backend Proxy	Frontend Credentials	Pre-Signed URLs
Security	High	CRITICAL RISK	High (Temporary)
Performance	Slow (Double hop)	Fast	Fast (Direct)
Scalability	Low (Server load)	High	High

Building the Bridge: A Quick Implementation Guide 

The backend generates the upload URL before the client starts the upload. Using the AWS SDK, the service creates a pre-signed URL for a specific S3 object and sets a short expiry, usually between 60 and 300 seconds. The backend returns this URL to the client, which then uploads the file directly to S3.  

Below is an example on how to generate PUT pre-signed URL using AWS SDK v3 (Node.js):

In a production setup, the backend runs with IAM roles attached to the compute service, such as EC2, ECS, or Lambda. These roles provide temporary credentials that AWS manages and rotates automatically. The service uses these credentials to generate pre-signed URLs with scoped permissions, typically limited to actions like uploading a single object. 

The Trade-offs: Navigating the Limitations 

While pre-signed URLs are powerful, they aren’t a “set it and forget it” solution. You need to account for these before moving to production. 

1. The “Use It or Lose It” Window: These links are strictly temporary. If the URL expires before the upload starts, the user will hit an error and need to request a new one. 

2. The Open-Key Risk: Think of the URL like a guest pass; anyone who has the link can use it. To stay secure, you must generate them only for authenticated users and keep the expiration window very short. 

3. Permission Dependency: If the AWS account that created the link loses its permissions, the URL stops working immediately, regardless of the expiration time. 

4. The Size Limit Challenge: Standard pre-signed URLs don’t let you cap the file size easily. A user could theoretically upload a massive file that exceeds your needs, without extra configuration (like “POST policies”). 

5. Time Sync Issues: If your server’s internal clock is out of sync with AWS, it might generate URLs that AWS thinks are already expired. 

Best Practices for Production Use 

Pre-signed URLs work well in production when you handle access, validation, and S3 configuration correctly. Gaps in any of these areas lead to failed uploads or unintended access. 

Access control 

The backend should generate URLs only for authenticated users and restrict them to a specific object path and action. Run the service with IAM roles attached to your compute environment so AWS manages and rotates credentials. This keeps access scoped and avoids long-lived secrets in code. 

Request validation 

The backend should validate file name, type, and expected size before generating the URL. These checks prevent unsupported formats and oversized uploads from reaching S3 and reduce downstream issues. 

S3 configuration 

S3 needs to accept the upload request exactly as the client sends it. Configure CORS (Cross-Origin Resource Sharing) to allow required methods from trusted origins. Align Object Ownership settings with your bucket configuration. Upload requests that include unsupported headers, such as ACLs in enforced ownership buckets, fail at runtime. 

Operational constraints 

• Keep expiry windows short to reduce reuse risk 

• Enable checksum validation to ensure file integrity 

• Maintain clock sync with AWS to prevent signature errors 

What changed: 

• Broke the front into sections immediately 

• Kept explanations short under each heading 

• Removed long opening paragraph 

• Still maintains cause → effect without sounding like a dump 

Advanced Use Case: Large Files and Multipart Uploads 

When files are larger than 100 MB, they’re more likely to experience interruptions during upload, making a single PUT pre-signed URL less reliable. A multipart upload is the recommended approach for handling large files, supporting uploads of up to 5 TB. The process is coordinated by your backend in three steps: 

1. Initiate the upload 
   The backend sends a CreateMultipartUpload request to Amazon S3, which returns a unique UploadId for the file.  

2. Generate pre-signed URLs for each part 
   The file is divided into smaller chunks (typically 5 MB or more). The backend creates a separate pre-signed URL for each chunk using UploadPartCommand and the UploadId, then sends the complete list of URLs to the frontend.  

3. Complete the upload 
   After all file parts are uploaded successfully, the frontend sends the corresponding ETags to the backend. The backend then calls CompleteMultipartUpload, allowing Amazon S3 to combine all the uploaded parts into a single file. 

This approach improves reliability and performance during large uploads: 

• Failed parts can be retried without restarting the entire upload  

• Parallel uploads reduce total upload time  

• The upload can resume if the UploadId is preserved  

Use this pattern when file sizes grow beyond typical request limits or when network reliability becomes a concern. 

Architecture for Growth: Why We Switched to Direct-to-S3 Uploads  

Pre-signed URLs are more than a technical optimization, they represent a scalable, cloud-native design pattern for modern applications handling user-generated content. By enabling direct uploads to S3, organizations can eliminate unnecessary backend bandwidth consumption, reduce infrastructure strain, and improve overall upload performance.  

When implemented with short expiration windows, least-privilege IAM roles, proper validation, checksum enforcement, and secure CORS policies, pre-signed URLs provide a secure and cost-efficient solution for file handling at scale. For large files, combining this approach with multipart uploads ensures resilience, parallelism, and improved reliability.  

For systems expecting growth in file size or upload volume, this architecture should be strongly considered as a best-practice design choice.