System design for GMAIL email system

Designing a system similar to Gmail involves creating a robust email platform that allows users to send and receive emails, manage their inbox, handle attachments, and integrate with various features like calendars and contacts. Below is a detailed system design using Prisma for database modeling, Express.js for API routes, and a recommended tech stack:

Tech Stack

• Backend: Node.js with Express.js
• Database: PostgreSQL for relational data (users, emails, contacts), Redis for caching
• ORM: Prisma for database interactions
• Caching: Redis for caching frequently accessed data (session management, email threads)
• Authentication: JWT (JSON Web Tokens) for user authentication and authorization
• Messaging: RabbitMQ for handling asynchronous tasks (email delivery, notifications)
• Storage: Amazon S3 for storing email attachments securely
• Monitoring: Prometheus and Grafana for monitoring system metrics

System Components

1. Client Applications (Web and Mobile):

   • Interfaces for users to compose and send emails, manage inbox, search for emails, manage contacts, and integrate with calendar features.

2. Express.js Backend

   • Prisma Models:

     // schema.prisma
     
     datasource db {
       provider = "postgresql"
       url      = env("DATABASE_URL")
     }
     
     generator client {
       provider = "prisma-client-js"
     }
     
     model User {
       id          Int      @id @default(autoincrement())
       username    String   @unique
       email       String   @unique
       password    String
       contacts    Contact[]
       sentEmails  Email[]
       receivedEmails Email[]
       createdAt   DateTime @default(now())
     }
     
     model Contact {
       id        Int    @id @default(autoincrement())
       userId    Int
       user      User   @relation(fields: [userId], references: [id])
       name      String
       email     String
       createdAt DateTime @default(now())
     }
     
     model Email {
       id            Int      @id @default(autoincrement())
       senderId      Int
       sender        User     @relation(fields: [senderId], references: [id])
       recipients    User[]
       subject       String
       body          String
       attachments   Attachment[]
       createdAt     DateTime @default(now())
     }
     
     model Attachment {
       id        Int      @id @default(autoincrement())
       emailId   Int
       email     Email    @relation(fields: [emailId], references: [id])
       filename  String
       filetype  String
       filesize  Int
       url       String
       createdAt DateTime @default(now())
     }
     

   • Express API Routes:

     // server.js
     
     const express = require('express');
     const { PrismaClient } = require('@prisma/client');
     const prisma = new PrismaClient();
     const jwt = require('jsonwebtoken');
     const bcrypt = require('bcrypt');
     const multer = require('multer');
     const path = require('path');
     const fs = require('fs');
     const { v4: uuidv4 } = require('uuid');
     
     const app = express();
     app.use(express.json());
     
     // Multer storage configuration for file uploads
     const storage = multer.diskStorage({
       destination: function (req, file, cb) {
         const uploadPath = path.join(__dirname, 'uploads');
         fs.mkdirSync(uploadPath, { recursive: true });
         cb(null, uploadPath);
       },
       filename: function (req, file, cb) {
         const uniqueSuffix = uuidv4();
         cb(null, `${file.fieldname}-${uniqueSuffix}-${file.originalname}`);
       },
     });
     
     const upload = multer({ storage });
     
     // Endpoint for user registration
     app.post('/register', async (req, res) => {
       const { username, email, password } = req.body;
       try {
         const hashedPassword = await bcrypt.hash(password, 10);
         const user = await prisma.user.create({
           data: {
             username,
             email,
             password: hashedPassword,
           },
         });
         res.json(user);
       } catch (error) {
         console.error(error);
         res.status(500).json({ error: 'Failed to register user' });
       }
     });
     
     // Endpoint for user login and JWT generation
     app.post('/login', async (req, res) => {
       const { username, password } = req.body;
       try {
         const user = await prisma.user.findUnique({ where: { username } });
         if (!user) {
           return res.status(404).json({ error: 'User not found' });
         }
         const passwordMatch = await bcrypt.compare(password, user.password);
         if (!passwordMatch) {
           return res.status(401).json({ error: 'Invalid password' });
         }
         const token = jwt.sign({ userId: user.id }, 'secret', { expiresIn: '1h' });
         res.json({ token });
       } catch (error) {
         console.error(error);
         res.status(500).json({ error: 'Login failed' });
       }
     });
     
     // Endpoint for sending an email
     app.post('/emails', upload.array('attachments'), async (req, res) => {
       const { userId, recipients, subject, body } = req.body;
       try {
         const sender = await prisma.user.findUnique({ where: { id: userId } });
         if (!sender) {
           return res.status(404).json({ error: 'Sender not found' });
         }
         const recipientUsers = await prisma.user.findMany({ where: { email: { in: recipients } } });
         const email = await prisma.email.create({
           data: {
             senderId: sender.id,
             recipients: { connect: recipientUsers.map((user) => ({ id: user.id })) },
             subject,
             body,
             attachments: req.files.map((file) => ({
               filename: file.originalname,
               filetype: file.mimetype,
               filesize: file.size,
               url: file.path,
             })),
           },
         });
         res.json(email);
       } catch (error) {
         console.error(error);
         res.status(500).json({ error: 'Failed to send email' });
       }
     });
     
     // Endpoint for fetching user's inbox
     app.get('/emails/inbox', async (req, res) => {
       const { userId } = req.query;
       try {
         const user = await prisma.user.findUnique({
           where: { id: parseInt(userId) },
           include: {
             receivedEmails: true,
           },
         });
         res.json(user.receivedEmails);
       } catch (error) {
         console.error(error);
         res.status(500).json({ error: 'Failed to fetch inbox' });
       }
     });
     
     const PORT = process.env.PORT || 3000;
     app.listen(PORT, () => {
       console.log(`Server is running on http://localhost:${PORT}`);
     });
     

3. Database Layer

   • PostgreSQL: Stores user data, email metadata, attachments, and transactional information.
   • Redis: Used for caching email threads, session management, and improving application performance.

4. Authentication and Authorization

   • JWT: Token-based authentication for securing API endpoints and managing user sessions.

5. Messaging

   • RabbitMQ: Used for asynchronous tasks such as email delivery, notifications, and scheduling reminders.

6. Storage

   • Amazon S3: Stores email attachments securely and provides efficient retrieval.

7. Monitoring and Analytics

   • Prometheus and Grafana: Monitor system metrics, track performance, and troubleshoot issues proactively.

Scalability and Fault Tolerance

• Horizontal Scaling: Deploy multiple instances of microservices and use load balancing to handle high traffic and ensure availability.
• Database Sharding: Partition databases to distribute load and scale horizontally as the number of users and emails increase.
• Redundancy and Backup: Store backups in Amazon S3 and deploy services across multiple availability zones (AZs) for fault tolerance and disaster recovery.

Security

• Data Encryption: Use SSL/TLS for secure data transmission and implement encryption at rest for sensitive information stored in databases.
• Authorization: Implement role-based access control (RBAC) to restrict access to sensitive APIs and operations.