Daily Mail Summarization using T5

Transformers

Submitted in partial fulfilment for the completion of Internship of BE-VII

Semester of

BACHELOR OF ENGINEERING

IN

INFORMATION TECHNOLOGY

Submitted by

L V Vishva Sree 160121737083

Under the guidance of

N Shiva Kumar
Assistant professor,Dept. of IT

Department of Information Technology

CHAITANYA BHARATHI INSTITUTE OF TECHNOLOGY

An Autonomous Institute,Affiliated to Osmania University

November,2024
 CERTIFICATE

This is to certify that the project titled Daily Mail Summarization using

T5 Transformers is carried out by L.V.Vishva Sree (160121737083) in

partial fulfillment of the requirements for the completion of Internship during
VII semester of Bachelor of Engineering in Information Technology in

the year 2024-25.

Mentor Head of the Department

N Shiva Kumar Dr. M Venu Gopalachari
Assistant professor,Dept.of IT Professor and Head, IT

Gandipet(V),Ranga Reddy (Dist.)–500075, Hyderabad, T.S.

www.cbit.ac.in
 Acknowledgement

The satisfaction that accompanies the successful completion of the task

would be incomplete without the mention of the people who made it possible,
whose constant guidance and encouragement crown all the efforts with success.

I express my sincere gratitude and profound appreciation for the dedicated

personal interest and invaluable guidance provided by our mentor, N. Shiva
Kumar, Assistant Professor.

I am particularly thankful to Dr. M Venu Gopalachari, the Head of

the Department, Department of Information Technology, his guidance, intense
support and encouragement, which helped me to mould our project into a
successful one.

I show gratitude to our honorable Principal Prof.C.V.Narasimhulu, for

providing all facilities and support.

I thank all the staff members of Information Technology department for

their valuable support and generous advice. Finally thanks to all our friends
and family members for their continuous support and enthusiastic help.

L.V.Vishva Sree(160121737083)

iii
 Abstract

In the age of information overload, individuals and organizations face the

challenge of processing vast amounts of textual data on a daily basis. To
address this, automated text summarization has emerged as a crucial tool to
help users quickly extract key insights from lengthy content. This project
develops an email summarization tool using the T5 (Text-to-Text Transfer
Transformer) model, a state-of-the-art transformer model capable of handling
various natural language processing tasks, including summarization. The goal
is to automate the process of summarizing email content, enabling users to
quickly grasp the essential information without reading through long emails.
The tool uses an abstractive summarization approach, where the T5 model
generates concise summaries by rephrasing and condensing the original content.
Unlike traditional extractive methods, which select sentences directly from the
input text, T5 generates summaries that are more coherent and natural,
making them ideal for summarizing emails, news articles, legal documents,
and customer support inquiries.
The project involves fine-tuning the pre-trained T5 model on the Daily
Mail summarization dataset, optimizing it for accurate and context-aware
summarization. The model is integrated into a web application using Flask,
providing a user-friendly interface where users can input email text or upload
documents for summarization. The application ensures seamless interaction,
enabling users to view and copy summaries with ease.
This system not only offers a solution for efficient email management but
also has broader applications in industries such as legal, news aggregation, and
customer support, where summarization can significantly improve information
processing. By leveraging advanced natural language processing techniques, this
project demonstrates the potential of transformer-based models in transforming
how we interact with and process large volumes of text data

iv
 Table of Contents

Title Page No.

Acknowledgement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iii
Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iv
List of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii
ABBREVIATIONS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viii
CHAPTER 1 INTRODUCTION . . . . . . . . . . . . . . . . . . . . 1
1.1 Need for Email Summarization . . . . . . . . . . . . . . . . . . . 1
1.2 Overview of the Project . . . . . . . . . . . . . . . . . . . . . . . . 3
1.3 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.3.1 Problem Description . . . . . . . . . . . . . . . . . . . . . . 3
1.3.2 Challenges in Email Summarization . . . . . . . . . . . . 4
1.4 The Emergence of Daily Mail Summarization . . . . . . . . . . . 5
1.5 Key Features of the Daily Mail Summarizer . . . . . . . . . . . 6
CHAPTER 2 EXISTING SYSTEM . . . . . . . . . . . . . . . . . . 8
CHAPTER 3 PROPOSED SYSTEM . . . . . . . . . . . . . . . . . . 14
3.1 System Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . 14
3.1.1 Components of the Proposed System . . . . . . . . . . . . 15
3.1.2 Functional Flow . . . . . . . . . . . . . . . . . . . . . . . . 17
3.1.3 Features of the Proposed System . . . . . . . . . . . . . . . 17
3.1.4 Technologies Used . . . . . . . . . . . . . . . . . . . . . . . 18
3.2 Technical Architecture . . . . . . . . . . . . . . . . . . . . . . . . . 19
CHAPTER 4 SYSTEM REQUIREMENTS . . . . . . . . . . . . . 20
4.1 Software Requirements . . . . . . . . . . . . . . . . . . . . . . . . . 20
4.1.1 Programming Languages: . . . . . . . . . . . . . . . . . . . 20
4.1.2 Libraries and Frameworks . . . . . . . . . . . . . . . . . . . 20
4.1.3 Dataset Daily Mail Summarization Dataset . . . . . . . . . 22
4.1.4 Web Framework and Hosting . . . . . . . . . . . . . . . . 22
4.2 Hardware Requirements . . . . . . . . . . . . . . . . . . . . . . . . 23
4.2.1 Development Hardware . . . . . . . . . . . . . . . . . . . . 23
4.2.2 Model Training Hardware . . . . . . . . . . . . . . . . . . 24
4.2.3 Deployment Hardware . . . . . . . . . . . . . . . . . . . . . 25
 4.3 Additional Tools and Platforms . . . . . . . . . . . . . . . . . . . 25
CHAPTER 5 IMPLEMENTATION AND RESULTS . . . . . . . 27
5.1 Project Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
5.2 Data Collection and Preparation . . . . . . . . . . . . . . . . . . 27
5.2.1 Importing libraries . . . . . . . . . . . . . . . . . . . . . . 27
5.3 Data Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
5.3.1 Text Preprocessing . . . . . . . . . . . . . . . . . . . . . . 29
5.3.2 Splitting and Tokenization . . . . . . . . . . . . . . . . . . 30
5.3.3 Model Setup and Training . . . . . . . . . . . . . . . . . . 31
5.3.4 Development of Summarization Function . . . . . . . . . . 32
5.4 Summarization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
5.5 Application Building . . . . . . . . . . . . . . . . . . . . . . . . . . 34
5.5.1 Building HTML pages . . . . . . . . . . . . . . . . . . . . 35
CHAPTER 6 CONCLUSION . . . . . . . . . . . . . . . . . . . . . . 37
CHAPTER 7 FUTURE SCOPE . . . . . . . . . . . . . . . . . . . . . 38
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
 List of Figures

3.1 Flowchart . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

5.1 The template folder contains HTML pages. The app.py contains
the flask code where the user gives input text/mail or can even
upload files for summarization . . . . . . . . . . . . . . . . . . . . 27
5.2 Import libraries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
5.3 Read the dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
5.4 Data Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
5.5 Text Processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
5.6 Splitting and Tokenization . . . . . . . . . . . . . . . . . . . . . . 31
5.7 Home page . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
5.8 Upload Page . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
5.9 Result Now Page . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

vii
 Abbrevations

Abbrevations Description

CBIT Chaitanya Bharathi Institute of Technology

IT Information Technology

AI Airtificial Intelligence

API Application Programming Interface

UI User Interface

SVM Support Vector Machine

viii
 CHAPTER 1

INTRODUCTION

In today’s digital world, individuals and organizations are overwhelmed

with an ever-increasing volume of emails, news articles, and other types of
textual content. The sheer amount of information available demands efficient
methods to process and digest content quickly. Traditional methods of reading
lengthy emails or documents can be time-consuming and inefficient. This has
led to the development of tools that can automatically summarize content,
enabling users to extract key information in a concise form. Automated text
summarization has become a key area in Natural Language Processing (NLP),
which aims to address this challenge by reducing the length of text while
preserving its essential meaning.
One of the most advanced models in NLP is T5 (Text-to-Text Transfer
Transformer), developed by Google Research. T5 has shown significant success
across a variety of NLP tasks, including summarization, translation, and
question-answering. By framing all tasks as ”text-to-text” problems, T5
can process input text and generate a coherent and contextually relevant
summary. This project utilizes T5’s summarization capabilities to build an
email summarization tool that allows users to quickly extract key points from
long emails.

1.1 Need for Email Summarization

Emails, particularly in business settings, often contain a large amount
of irrelevant or repetitive content that can easily be filtered out. However,
manually sifting through these emails is time-consuming, and individuals can
miss important information in the process. Moreover, email communication
spans a variety of contexts—whether it’s a client inquiry, a project update,
or internal communication—making it challenging to summarize effectively
without missing key points.

1
 The growing demand for time-efficient communication solutions has fueled
the need for email summarization tools. These tools can help reduce the reading
load by automatically extracting the most relevant content from emails and
presenting it in a concise, readable format.

• Job Seekers: For those applying for jobs, having a resume that high-
lights the most relevant skills and experiences is crucial. However,
creating the perfect summary or tailoring resumes for different roles can
be an exhausting and time-consuming task. An AI-powered summarizer
can assist by automatically identifying key skills, qualifications, and ex-
periences, ensuring that the most important information is presented in
a clear, concise format.

• Researchers and Students: Researchers and students often deal with

large volumes of academic papers, research articles, and other scholarly
documents. Reading these documents in their entirety can be overwhelm-
ing and may divert valuable time from actual research. An effective
document summarization tool can help by condensing long articles into
brief yet accurate summaries, highlighting the core arguments, findings,
and conclusions, making it easier to quickly evaluate the relevance of a
paper without reading it cover to cover.

• Professionals: Legal documents, such as contracts, agreements, and case

law, are often dense and filled with complex language. Lawyers and par-
alegals need to understand the key clauses and terms in these documents
quickly. Automated summarization can help legal professionals identify
critical information faster and with more consistency, allowing them to
focus their time on higher-level tasks such as analysis and strategy.

• Businesses and Professionals: Professionals across industries are re-

quired to read and interpret large amounts of information. Whether it’s
business reports, market analyses, or project proposals, the ability to
quickly extract insights from these documents is vital for decision-making.
Summarization tools can assist by generating concise overviews, ensuring

Department of Information Technology 2

 that professionals can quickly access the most relevant information and
make informed decisions.

These scenarios illustrate the need for an intelligent, automated summa-

rization tool that can handle different types of documents, accurately
process complex information, and present concise summaries without
losing critical context.

1.2 Overview of the Project

This project’s objective is to develop an email summarization tool that
leverages the T5 model. By fine-tuning this transformer-based model on a
dataset specifically focused on summarization (e.g., the Daily Mail Summa-
rization Dataset), we aim to generate summaries that are both coherent and
contextually accurate.
Furthermore, we will build a web-based interface using Flask, allowing users
to interact with the system. Users will be able to input email content (either
by pasting text or uploading files) and get back a summary generated by the
T5 model. The project will highlight the power of modern NLP techniques,
specifically abstractive summarization, which goes beyond simple extraction
and generates more natural summaries.

1.3 Problem Statement

1.3.1 Problem Description

The rapid growth of digital communication has resulted in an exponential
increase in the amount of content being shared. As email and messaging
systems have become central to business communication, users are often
overwhelmed by long, dense emails filled with excessive detail. While traditional
methods of summarization, such as extractive summarization, focus on selecting
parts of the text verbatim, they fail to offer an intelligent summary that
captures the core meaning in a cohesive manner.
In addition, existing email summarization tools tend to either over-simplify

Department of Information Technology 3

or distort important information. They may miss crucial points, or the
summary may be too generic to be useful in specific contexts, such as legal,
technical, or business emails.
Furthermore, many summarization systems are designed only for general
content, lacking the ability to capture nuances present in different domains.
For example, an email requesting approval for a budget may require different
handling than one containing meeting minutes.

1.3.2 Challenges in Email Summarization

• Content Diversity:Emails often span multiple topics and can include
formal language, technical jargon, or a mix of personal and professional
tones. This makes it difficult for summarization systems to identify and
preserve the most important elements of the message.

• Inconsistent and Subjective Results: Manual summarization is highly

dependent on the person performing it. Different individuals may focus
on different aspects of a document, leading to subjective summaries. For
example, two people summarizing the same report may include different
details or omit essential information, depending on their interpretation
of what is important.

• Inadequate Handling of Complex Documents: Many existing auto-

mated summarization tools, particularly those based on older AI models,
struggle with complex documents. These tools often fail to capture
nuanced information, especially in documents containing specialized vo-
cabulary or technical language. As a result, summaries generated by
these tools may be overly simplistic, inaccurate, or incomplete, which
undermines their usefulness.

• Limited File Format Support: Traditional summarization tools often

only support certain file formats (such as plain text), making them
inconvenient for users who need to summarize documents in formats
like PDFs, Word documents, or scanned images. This limitation can be

Department of Information Technology 4

 especially frustrating for professionals who work with varied document
types and need a flexible, all-in-one summarization tool.

• Lack of User-Friendliness: Many AI-powered summarization tools are

complex and require a certain level of technical expertise to use effectively.
For example, some tools may require users to input code or manually
adjust parameters to get the best results. This lack of user-friendliness
makes it difficult for non-technical users to take advantage of these tools.

These challenges highlight the need for a solution that combines the power
of AI with ease of use, accuracy, and flexibility.

1.4 The Emergence of Daily Mail Summariza-

tion
The emergence of Daily Mail Summarization is a significant milestone in
the field of natural language processing (NLP), particularly in abstractive
text summarization. The Daily Mail Summarization Dataset, which pairs
news articles with human-generated summaries, has become a key resource
for training machine learning models to condense lengthy articles into concise,
meaningful summaries. The dataset’s diversity, covering various topics such
as politics, health, and entertainment, provides a rich foundation for models
to generate summaries that are both relevant and fluent. The introduction of
advanced transformer models like T5 (Text-to-Text Transfer Transformer) has
further enhanced the capabilities of summarization tools. T5’s ability to treat
all NLP tasks as text-to-text problems allows it to effectively summarize news
articles, generate coherent abstractions, and produce summaries that resemble
human-written content.
T5’s success in summarizing Daily Mail articles has opened doors for
real-world applications in news aggregation platforms, content curation, and
legal or business document processing. By fine-tuning T5 on the Daily Mail
Summarization Dataset, it is possible to generate highly accurate and contex-
tually relevant summaries that can be used for faster information consumption.

Department of Information Technology 5

News platforms like Google News and Flipboard, as well as content creators
and businesses, can benefit from these summarization techniques by providing
users with quick, personalized summaries of articles. As summarization models
continue to evolve, they offer an opportunity to improve information retrieval
and processing across multiple domains, transforming the way we interact with
text-based data.

1.5 Key Features of the Daily Mail Summarizer

The Daily Mail Summarizer offers several key features that make it a
powerful tool for automating the summarization of news articles, legal docu-
ments, emails, and other lengthy textual content. These features ensure the
tool’s relevance across multiple domains such as news aggregation, business,
and customer service.

• Abstractive Summarization: The tool uses advanced abstractive

summarization techniques, particularly with the T5 model, to generate
summaries that are not just extracts of key sentences but also rephrase
and condense the original content into shorter, more coherent forms.
This approach ensures that the summaries retain the original meaning
while presenting it in a concise and fluid manner.

• Domain Flexibility: While primarily trained on news articles from the

Daily Mail dataset, the summarizer can adapt to a variety of domains.
It can generate summaries for articles on diverse topics such as politics,
health, finance, and entertainment, making it versatile and applicable in
different industries, including legal and customer service contexts.

• High-Quality Summaries: The use of a powerful transformer model

like T5 (Text-to-Text Transfer Transformer) ensures that the summaries
generated are of high quality. These summaries are contextually accurate,
coherent, and relevant, closely mimicking human-written summaries in
their fluency and brevity.

• Fast Processing and Scalability: The summarizer is designed to

Department of Information Technology 6

 process large volumes of text efficiently. Whether used for summarizing
a single document or a collection of articles, the system can scale
to handle significant amounts of data quickly, making it suitable for
applications in news aggregation platforms, legal document review, or
enterprise content management.

• Customizable Length and Content Focus: The summarizer can be

adjusted to create summaries of varying lengths depending on the user’s
needs. It can also prioritize key themes and concepts, ensuring that the
most important aspects of the content are captured in the summary,
which is critical for scenarios like news aggregation or customer support.

• Integration with Web Platforms: The summarization tool can be

integrated into web applications via frameworks like Flask, allowing easy
access for users to upload documents or input text for summarization.
This web integration provides an intuitive and accessible way for users
to interact with the tool, making it ideal for deployment in real-time
applications such as customer service, news platforms, or enterprise-level
tools.

• User-Friendly Interface: The summarizer features a simple, user-

friendly interface where users can either upload files or paste text to be
summarized. With features such as drag-and-drop file upload and real-
time progress indicators, users can quickly and easily obtain summaries
without needing technical expertise.

Department of Information Technology 7

 CHAPTER 2

EXISTING SYSTEM

Mail summarization has become a pivotal task in the field of Natural

Language Processing (NLP), especially with the growing amount of data and
information available. The need for systems that can effectively condense large
volumes of text into digestible summaries has driven the development of various
summarization techniques and tools. Summarization systems can broadly be
categorized into two types: extractive** and abstractive. This section explores
existing systems in the context of these two techniques, focusing on their
functionality, benefits, and limitations.

• 1.Extractive Summarization Systems

Extractive summarization systems operate by selecting a subset of the

input text (usually sentences) and combining them to form a summary.
These systems aim to retain the most important information from the
original text by extracting entire sentences or phrases directly. Extractive
methods are simpler to implement and often deliver faster results, but
they may fail to generate coherent or fluent summaries since they don’t
rephrase the selected content.

– a.TextRank Algorithm

One of the most popular extractive summarization techniques is

the TextRank algorithm, introduced by Radev et al. in 2004.
This algorithm is based on the principles of PageRank, used by
search engines like Google. It works by constructing a graph where
the nodes are sentences, and edges represent semantic relationships
between them. TextRank then ranks the sentences based on their
importance within the context of the entire document and selects
the top-ranked sentences to form a summary.

∗ Advantages:

8
 · 1.Simple to implement and computationally efficient.

· 2. Does not require labeled data or deep learning models,

making it easy to deploy on a variety of datasets.

∗ Limitations:

· 1.May produce summaries that lack coherence, as the sen-

tences are selected based on their importance rather than
their contribution to a fluent narrative.

· 2.Relies heavily on sentence-level relationships, which may

not always align with how human summarizers perceive the
text.

– b.Sumy

Sumy is an open-source Python library that supports multiple ex-

tractive summarization algorithms, including TextRank, Lsa (Latent
Semantic Analysis), and LexRank. It allows users to quickly sum-
marize text with minimal setup. Sumy is primarily useful for
individuals who want to apply extractive summarization methods
without delving deep into the underlying algorithms.

∗ Advantages:

· 1.Easy-to-use and accessible to users who do not have a

strong background in NLP.

· 2.Offers a variety of summarization techniques, allowing users

to experiment and choose the best algorithm for their data.

∗ Limitations:

· 1.The quality of the summary may suffer, as extractive

methods do not always capture the key points effectively.

· 2.The system can struggle with non-textual data like images

or multimedia content.

• 2.Abstractive Summarization Systems

Unlike extractive summarization, abstractive summarization generates new

sentences that convey the meaning of the original text. This approach

Department of Information Technology 9

 is more akin to how humans summarize text, as it involves paraphrasing
and restructuring the content. Abstractive summarization systems typi-
cally rely on deep learning models, such as Recurrent Neural Networks
(RNNs), Long Short-Term Memory (LSTM) networks, or Transformer-
based models.

– a.OpenAI’s GPT-3

OpenAI’s GPT-3 is one of the most advanced transformer-based

models for NLP tasks, including summarization. GPT-3 has shown
remarkable performance in generating fluent and coherent summaries
due to its large-scale pre-training on vast amounts of text data. Un-
like extractive summarization systems, GPT-3 can generate new text,
making it capable of producing high-quality, human-like summaries.

GPT-3 is a generative model, meaning it produces novel text based

on the input prompt. For summarization, it can take an article or
a document as input and output a summary that is fluent, concise,
and well-structured. The model’s ability to understand context
allows it to generate summaries that focus on the most relevant
information.

∗ Advantages:

· 1.High-quality, human-like summaries that reflect a deep

understanding of the text.

· 2.Can handle a wide range of input text, from news articles

to legal documents.

∗ Limitations:

· 1. Requires significant computational resources to run, mak-

ing it expensive for widespread use.

· 2. While GPT-3 is proficient in generating fluent sum-

maries, it can sometimes hallucinate or produce incorrect
information, especially with ambiguous inputs.

– b.Google’s T5 (Text-To-Text Transfer Transformer)

Department of Information Technology 10

 Google’s T5 model, which stands for Text-to-Text Transfer Trans-
former, has revolutionized abstractive summarization. T5 treats
every NLP task as a ”text-to-text” problem, meaning that both the
input (e.g., an article) and output (e.g., a summary) are treated
as text. This approach simplifies the model’s architecture and al-
lows it to be fine-tuned for various tasks, including summarization,
translation, and question answering.

The T5 model is particularly useful for summarization because it

is pre-trained on vast datasets and can be fine-tuned on specific
domains, such as news articles or legal documents, to generate more
targeted summaries. When fine-tuned on specific datasets, such
as the **Daily Mail Summarization Dataset**, T5 has shown to
produce summaries that are both concise and contextually relevant.

∗ Advantages:

· 1.High-quality summaries that are not just extractive but

also semantically accurate and fluent.

· 2.The text-to-text approach makes it versatile for a wide

range of NLP tasks.

∗ Limitations:

· 1.Training large models like T5 requires substantial compu-

tational power.

· 2.Fine-tuning requires a labeled dataset, which might not

always be available or easy to obtain.

– c.BART (Bidirectional and Auto-Regressive Transformers)

Another noteworthy model for abstractive summarization is BART,

developed by Facebook AI. BART combines the benefits of both
bidirectional and autoregressive models, making it highly effective for
tasks such as summarization. It works by corrupting the input text
(e.g., by randomly masking words) and then learning to reconstruct
the text, which enables it to generate coherent summaries even when
the input contains noise or irregularities.

Department of Information Technology 11

 BART has been shown to outperform many traditional models in
tasks like abstractive summarization, thanks to its bidirectional
encoder and autoregressive decoder, which can better capture both
local and global dependencies in text.

∗ Advantages:

· 1.Produces high-quality, human-like summaries.

· 2.Can handle noisy data and incomplete text more effectively

than many other models.

∗ Limitations:

· 1.Requires considerable computational resources for training.

· 2.Like T5, it may generate incorrect or nonsensical informa-

tion if the input is ambiguous or if fine-tuning is not done
carefully.

– 3.Hybrid Summarization Systems

Some existing systems combine both extractive and abstractive meth-

ods to enhance the quality of summaries. These hybrid systems
typically use extractive summarization to identify key sentences or
topics and then apply an abstractive summarization model to refine
the content and produce a more coherent, human-readable summary.

∗ a.PEGASUS (Pre-training with Extracted Gap-sentences

for Abstractive Summarization)
PEGASUS is a model developed by Google that specifically tar-
gets abstractive summarization. It has achieved state-of-the-art
performance in several benchmarks by pre-training the model
with tasks that involve predicting missing sentences in a docu-
ment, thereby making it highly effective in understanding text
and generating relevant summaries. PEGASUS’s architecture
can be fine-tuned for a variety of tasks, making it adaptable
for different summarization challenges.

· Advantages:

Department of Information Technology 12

 · 1.Achieves state-of-the-art performance in abstractive sum-
marization benchmarks.

· 2.Handles both long and complex documents well.

· Limitations:

· 1.Requires large-scale pre-training and fine-tuning, which can

be computationally expensive.

· 2.Like other large transformer models, PEGASUS can some-

times generate summaries with hallucinated information.

The field of text summarization has evolved significantly over

the years, with several existing systems providing different ap-
proaches to generating summaries. While extractive summariza-
tion remains a straightforward and efficient option for producing
quick summaries,abstractive summarization models, such as T5,
GPT-3, and BART, have raised the bar by offering fluent,
human-like summaries that better capture the essence of a text.
Hybrid models like PEGASUS are also pushing the boundaries
of summarization quality by combining the strengths of both
extractive and abstractive methods.
However, these systems are not without their challenges. Compu-
tational requirements for training and fine-tuning large models,
such as T5 or GPT-3, remain a significant barrier to entry
for many developers and organizations. Furthermore, despite
their impressive capabilities, even the best abstractive models
can sometimes produce inaccurate or nonsensical summaries,
especially when confronted with ambiguous or complex text.

Department of Information Technology 13

 CHAPTER 3

PROPOSED SYSTEM

The proposed system aims to develop an email summariza-

tion tool leveraging the T5 (Text-to-Text Transfer Transformer)
model, specifically fine-tuned for summarizing long email con-
tent. The system will be designed to provide concise, accurate
summaries by extracting the most critical information from
email bodies, allowing users to quickly grasp the essential con-
tent without reading the entire text. This solution can be
applied to various real-world scenarios like news aggregation,
legal document summarization, and customer support, among
others.
The key objective is to create a user-friendly web-based tool that
utilizes advanced NLP techniques (Natural Language Process-
ing) to provide automatic email summarization. By fine-tuning
the pre-trained T5 model, this tool will be able to generate
summaries that maintain coherence and clarity while focusing
on the most relevant content. Additionally, the system will be
accessible via a web interface built using Flask, where users can
input their email content or upload files for processing.

3.1 System Architecture

The architecture of the proposed system consists of three key
components:

· 1.User Interface (UI):

A simple and interactive front-end where users can paste

or upload emails for summarization.

· 2.Backend (Model Integration):

14
 The T5 Transformer model is fine-tuned on the Daily Mail
Summarization Dataset to generate high-quality summaries.
The backend processes the input text, invokes the trained
model, and generates a summarized output.

· 3. Web Server:

The backend is integrated with a Flask web framework,

which handles HTTP requests, serves the input form for
email content, processes the text, and displays the generated
summary to the user.

3.1.1 Components of the Proposed System

· 1.Data Collection and Preprocessing:

· a.Dataset:

The system will use the Daily Mail Summarization Dataset

(available on Kaggle) for training the T5 model. This
dataset contains large amounts of text data paired with
summaries, ideal for fine-tuning.

· b.Text Preprocessing:

Text preprocessing is essential for cleaning the raw email

data. This includes steps such as tokenization, removing
stop words, punctuation, and non-relevant characters, and
splitting data into training and testing sets.

· 2. Model Development:

· a.Fine-Tuning the T5 Model:

The core of the system will be the T5 Transformer,

which will be pre-trained on the Daily Mail Summariza-
tion Dataset. We will fine-tune the model on this dataset,
adapting it for the email summarization task. This step
includes adjusting hyperparameters such as learning rate,
batch size, and the number of epochs.

Department of Information Technology 15

 · b.Model Evaluation:
After fine-tuning, the model’s performance will be evaluated
using metrics such as ROUGE (Recall-Oriented Understudy
for Gisting Evaluation), which measures the quality of the
generated summaries compared to human-generated refer-
ences.

· 3.Model Deployment:

· a.Flask Web Application:

The final step will involve integrating the trained model
into a Flask-based web application. The user interface (UI)
will allow users to enter email content or upload email files.
Once the user submits the input, the backend will process
the text through the fine-tuned T5 model and display the
summarized output.

· b.Front-End Design:
The web interface will be designed with minimal complexity,
offering a simple input box for users to paste text or a
file upload option. The output will be displayed clearly,
showing the summarized content.

· 4. Testing and Optimization:

· a.Testing:
After deploying the system, it will be tested for various edge
cases, such as extremely long emails, emails with mixed
content (e.g., tables, images, or non-text elements), and
emails with complex language. The system’s performance
will be monitored for consistency and reliability.

· b.Optimization:
Based on test results, the system will undergo optimiza-
tion, including fine-tuning the model further, improving the
UI/UX, and ensuring that the backend can handle large
datasets without significant delays.

Department of Information Technology 16

 3.1.2 Functional Flow

· 1.User Input:
The user visits the web application, either pasting email
content into a text box or uploading an email file.

· 2.Preprocessing:
The input is passed through preprocessing steps (such as
tokenization and cleaning) to prepare it for the model.

· 3.Summarization:
The preprocessed input is fed into the T5 model, which
generates a summary based on the context and content of
the email.

· 4.Displaying the Result:

The generated summary is displayed on the UI in a user-
friendly manner, allowing the user to read the concise version
of the email.

· 5.Further Interaction:
The user may choose to submit another email or perform
additional actions.

3.1.3 Features of the Proposed System

· 1.Automated Summarization:
The system will automatically generate summaries of emails,
reducing the time and effort required to read lengthy emails.

· 2.NLP-based Techniques:
The system uses state-of-the-art transformer models, ensur-
ing high-quality and contextually relevant summaries.

· 3.File Upload Option:

Users can upload email files (such as .txt or .pdf formats)
for summarization.

· 4.Real-time Results:

Department of Information Technology 17

 The system will provide real-time summaries, enabling users
to quickly get insights from long emails.

· 5.User-Friendly Interface:
The interface will be intuitive and accessible, requiring min-
imal user interaction to achieve optimal results.

· 6.Customization Options:
Users will be able to specify the length of the summary
(e.g., concise or detailed) or the level of abstraction desired.

3.1.4 Technologies Used

· 1.T5 Transformer Model:

A transformer-based model pre-trained by Google for NLP
tasks such as summarization.

· 2.Hugging Face Transformers Library:

A popular library that provides easy access to pre-trained
models, including T5, and tools for fine-tuning.

· 3.Python:
The primary programming language for implementing the
system, used with libraries like Flask, PyTorch, and Trans-
formers.

· 4.Flask:
A lightweight web framework for Python that will host the
system and handle user requests.

· 5.HTML/CSS:
For creating the front-end user interface.

· 6.JavaScript:
For enhancing the interactivity of the web pages.

Department of Information Technology 18

 3.2 Technical Architecture

Figure 3.1: Flowchart

Department of Information Technology 19

 CHAPTER 4

SYSTEM REQUIREMENTS

In the development and deployment of the Daily Mail Sum-

marization System using T5 Transformers, certain software and
hardware configurations are required to ensure smooth execu-
tion, efficient performance, and optimal results. The following
sections outline the essential software and hardware requirements
needed for the project.

4.1 Software Requirements

The software components for this project consist of several essen-
tial packages and libraries, along with a suitable web framework
to integrate the model and deploy it. The software stack re-
quired for the proposed system is as follows:

4.1.1 Programming Languages:

· Python 3.x:

· 1.Python is the primary programming language used for

implementing the T5 model, data preprocessing, training,
and the backend of the web application. It is widely used in
natural language processing tasks due to its robust libraries
and frameworks.

· 2.Recommended Version: Python 3.7 or higher.

4.1.2 Libraries and Frameworks

To build and train the model, as well as to integrate the

system into a web application, the following Python libraries

20
 are essential:

· Transformers (by Hugging Face):

· 1.This library provides pre-trained transformer models like
T5, which is used for text summarization. The library offers
a simple API to fine-tune models on custom datasets and
generate summaries.
· 2.Command for installation: pip install transformers

· Torch (PyTorch):
· 1.PyTorch is a deep learning framework required to work
with the T5 model, as it allows for GPU acceleration and
provides tools to train and fine-tune neural networks.
· 2.Command for installation: pip install torch

· Flask:
· 1.Flask is a micro web framework for Python that will
be used to create the web application. It handles routing,
rendering HTML templates, and managing user requests.
· 2.Command for installation: pip install flask

· Pandas:
· 1.Pandas is essential for reading, manipulating, and prepro-
cessing data. It is used for handling the Daily Mail dataset,
cleaning, and splitting it into training and testing sets.
· 2.Command for installation: pip install pandas

· NLTK (Natural Language Toolkit):

· 1.NLTK provides various tools for text preprocessing, such
as tokenization, stop word removal, and lemmatization. It
will be used to clean and prepare email content before
passing it through the model.
· 2.Command for installation: pip install nltk

· NumPy:
· 1.NumPy is a package for scientific computing in Python
and will be used for array manipulation and mathemati-

Department of Information Technology 21

 cal operations, which are frequently required during text
processing and model training.
· 2.Command for installation: pip install numpy

· Scikit-learn:
· 1.Scikit-learn is a machine learning library that will be useful
for model evaluation (e.g., calculating ROUGE scores) and
splitting datasets for training and testing purposes.
· 2.Command for installation: pip install scikit-learn

· HTML/CSS/JavaScript:
HTML, CSS, and JavaScript will be used to create and
style the user interface for the web application. JavaScript
can also be used for enhancing the interactivity of the web
pages.

4.1.3 Dataset Daily Mail Summarization Dataset

The Daily Mail Summarization Dataset is a collection of news

articles paired with human-written summaries. It is ideal for
training the T5 model for the summarization task. The dataset
can be downloaded from Kaggle at the following link: Daily
Mail Summarization Dataset.

4.1.4 Web Framework and Hosting

· Flask:
· 1.As mentioned, Flask will be used to handle user input,
display results, and serve the web application.
· 2.Flask Extensions: For deployment, extensions like Flask-
SocketIO (for real-time interaction) and Flask-WTF (for
form handling) may be used.

· Web Server:
· 1.Apache or Nginx can be used for deploying the application
in a production environment.

Department of Information Technology 22

 · 2.Alternatively, the system can be run on a local devel-
opment server during testing and integration (using Flask’s
default development server).

4.2 Hardware Requirements

The hardware requirements for running the system depend
largely on the dataset size, model training, and inference work-
load. The system can be deployed on various hardware config-
urations, with different requirements for development, training,
and deployment.

4.2.1 Development Hardware

During the development phase, the requirements are relatively

modest, especially for tasks like building the web interface,
preprocessing data, and deploying the web server. For basic
development, the following hardware configuration will suffice:

· Processor:

· 1.Intel i5 or i7 (or equivalent) with at least 4 cores.

· 2.For more complex tasks, like model training and fine-

tuning, a higher-end CPU with multiple cores (e.g., Intel i9
or AMD Ryzen 7/9) will speed up the process.

· RAM:

· 1.8GB or higher of RAM will be needed for smooth per-

formance during development.

· 2.For training large models, 16GB or more of RAM is

recommended.

· Storage:

· 1.A minimum of 10GB of free storage for code, datasets,

and intermediate files.

· 2.For large datasets, SSD (Solid State Drive) with faster

Department of Information Technology 23

 read/write speeds will significantly improve data processing
time.

· Graphics Card (GPU):

· 1.A GPU is not strictly necessary for the development

phase, but it is recommended for training large models like
T5.

· 2.NVIDIA GPU (e.g., GTX 1060, 1070, or RTX series) is

ideal for deep learning tasks, as it offers excellent support
for CUDA acceleration.

4.2.2 Model Training Hardware

The training phase will benefit significantly from using a pow-

erful GPU. Training a large model like T5 requires substantial
computational power, so the following hardware is recommended
for efficient model training:

· Processor:

· 1.Intel i7 or i9 (or equivalent) with 8 or more cores for fast

computation.

· 2.Multiple CPU cores are particularly useful for data pro-

cessing and parallel tasks.

· RAM:

· 1.16GB or more of RAM is highly recommended for handling

large datasets and model training.

· GPU:

· 1.For fine-tuning the T5 model, a powerful GPU is essential.

A NVIDIA RTX 2080, RTX 3080, or NVIDIA A100 would
be ideal for faster model training.

· 2.CUDA support is crucial for running deep learning models

efficiently on NVIDIA GPUs.

· Storage:

Department of Information Technology 24

 · 1.500GB or more of storage, preferably SSD, for storing
datasets, trained models, and intermediate files.

· 2.External storage or cloud-based storage options (e.g., AWS

S3, Google Cloud Storage) may be used for large datasets.

4.2.3 Deployment Hardware

For deployment, the system can be hosted on a cloud platform

or an on-premise server. The recommended hardware for hosting
the web application is:

· Processor:
A cloud server with at least 2-4 CPU cores (for medium-sized
deployments). For high availability, multi-core instances
should be considered.

· RAM:
8GB or more of RAM for the server hosting the web
application.

· Storage:
SSD storage for fast access to the model and data, around
10-20GB depending on the deployment scale.

· Network:
A stable internet connection with sufficient bandwidth for
real-time interactions between users and the server.

4.3 Additional Tools and Platforms

· Kaggle: To download the dataset, Kaggle offers an easy
interface for dataset access and exploration.

· Google Colab or Jupyter Notebooks: For prototyping

and training models in the early stages of development,
these platforms offer GPU access and easy setup.

· Cloud Platforms (Optional):

Department of Information Technology 25

 · AWS EC2: For scalable computing resources.

· Google Cloud: Offers cloud-based VMs and storage for

model hosting and training.

· Heroku: A platform for easy deployment of small-scale

applications.

Department of Information Technology 26

 CHAPTER 5

IMPLEMENTATION AND RESULTS

5.1 Project Structure

Figure 5.1: The template folder contains HTML pages. The app.py contains
the flask code where the user gives input text/mail or can even upload files
for summarization

5.2 Data Collection and Preparation

5.2.1 Importing libraries

There are many popular open sources for collecting the data.
Eg: kaggle.com, UCI repository, etc.
In this project, we have used .csv data. This data is downloaded
from kaggle.com. Please refer to the link given below to down-
load the dataset. Link: https://www.kaggle.com/datasets/evilspirit05/daily-
mail-summarization-dataset As the dataset is downloaded. Let
us read and understand the data properly with the help of some
visualisation techniques and some analysing techniques. Note:
There are several techniques for understanding the data. But

27
 here we have used some of it. In an additional way, you can
use multiple techniques.

Figure 5.2: Import libraries

Figure 5.3: Read the dataset

5.3 Data Analysis

As we have understood how the data is, let’s pre-process the
collected data. The download data set is not suitable for
training the machine learning model as it might have so much
randomness so we need to clean the dataset properly to fetch

Department of Information Technology 28

 good results. This activity includes the following steps. Keeping
the unique sentences:

Figure 5.4: Data Analysis

5.3.1 Text Preprocessing

· Text pre-processing (python packages) Text prepro-

cessing is a crucial step in Natural Language Processing
(NLP) and Information Retrieval (IR) tasks. The goal is to
convert raw text into a more meaningful and manageable
representation for further analysis.
In Python, several packages provide support for text pre-
processing operations. Some of the most common ones are:

· NLTK (Natural Language Toolkit) It is one of the most

widely used NLP libraries in Python. It provides tools for
tokenization, stemming, lemmatization, stop-word removal,
and more.

Department of Information Technology 29

 Figure 5.5: Text Processing

· Data preprocessingIt is a critical step in preparing the

text data for input into the T5 model for summarization.
The goal of preprocessing is to clean, standardize, and
format the text data so that the model can effectively
generate accurate and coherent summaries. Below is a
detailed explanation of the data preprocessing steps.

5.3.2 Splitting and Tokenization

· 1.Split the dataset into training, validation, and test sets

to evaluate model performance

· 2.Tokenize the cleaned text using the T5 tokenizer, ensuring

that text is appropriately truncated or split for model input.

Department of Information Technology 30

 Figure 5.6: Splitting and Tokenization

5.3.3 Model Setup and Training

· 1.Choose the appropriate T5 model variant (e.g., t5-small,

t5-base) based on the project’s computational resources and
requirements.

· 2.Experiment with different hyperparameters such as learn-

ing rate, batch size, and number of training epochs to
optimize model performance.

· 3.Evaluate the fine-tuned model on the validation set using

metrics such as ROUGE scores to assess the quality of the
generated summaries.

Department of Information Technology 31

 5.3.4 Development of Summarization Func-
tion

· 1.Develop a function to process input text through the T5

model and generate summaries.

· 2.Ensure the function can handle large volumes of data

efficiently

Department of Information Technology 32

 5.4 Summarization
The T5 (Text-To-Text Transfer Transformer) model, developed
by Google Research, is a versatile language model that frames
all NLP tasks as a text-to-text problem. This approach allows
tasks such as translation, summarization, and question answering
to be handled by a single model architecture. T5 is pre-trained
on a large corpus of text and can be fine-tuned for specific
tasks, like summarization.

Department of Information Technology 33

 5.5 Application Building
In this section, we will be building web application pages where
the user can find interacting options and the user must navigate

Department of Information Technology 34

 to the capturing button so that the camera will be turned on.
After the camera is turned on, the faces will be detected and
the captured image will be stored in the device of the user.
This section has the following tasks

· 1.Building HTML Pages

· 2.Building server-side script

5.5.1 Building HTML pages

Figure 5.7: Home page

Figure 5.8: Upload Page

Department of Information Technology 35

 Figure 5.9: Result Now Page

Department of Information Technology 36

 CHAPTER 6

CONCLUSION

The Daily Mail Summarization System using T5 Transformers is

a comprehensive solution that leverages the power of advanced
Natural Language Processing (NLP) and Machine Learning (ML)
techniques for the automatic summarization of lengthy email con-
tent. By using the T5 Transformer model, the system effectively
processes large amounts of unstructured text, transforming it
into concise summaries that capture the essential information.
This project demonstrates the capabilities of transformer-based
models in simplifying text-heavy tasks, such as summarization,
which can be applied in real-world scenarios like email summa-
rization, news aggregation, legal document review, and customer
support.
The model was fine-tuned using a Daily Mail Summarization
Dataset, which allowed it to generate meaningful and relevant
summaries of emails with high accuracy. The integration of
this model into a web application via Flask enables users to
interact with the system effortlessly, providing a streamlined
user experience. Throughout the project, various steps including
data collection, preprocessing, model training, and deployment
were executed, showcasing the entire pipeline of an AI-based
summarization system.
In conclusion, the system proves the effectiveness of T5 Trans-
formers in simplifying text data processing and demonstrates a
robust use case in the field of email summarization. It also
highlights the potential for transforming long-form content into
actionable summaries across various industries, thereby saving
time and improving productivity for users.

37
 CHAPTER 7

FUTURE SCOPE

While the Daily Mail Summarization System developed in this

project is effective and efficient, there are several areas where the
system can be enhanced and expanded to improve its accuracy,
scope, and usability. The future scope of this project includes
the following potential directions:

· Improving Model Performance

· 1.Fine-Tuning with Domain-Specific Data: The system
currently uses a general summarization dataset. To improve
its performance in specific industries, the model can be
fine-tuned using domain-specific datasets. For example, in-
tegrating datasets for legal contracts, medical records, or
customer support conversations can make the summariza-
tion system more specialized and effective in these areas.
· 2.Handling Multilingual Text: Currently, the model is
trained to summarize English text. Future improvements
can include extending its capabilities to support multiple
languages. This can be achieved by training or fine-tuning
multilingual versions of the T5 model, such as mBART
or mT5, to allow users to summarize emails in various
languages

· Optimizing for Speed and Efficiency

· 1.Model Optimization for Real-Time Summarization:
In a production environment, real-time summarization is
essential for user satisfaction. Therefore, optimizing the T5
model for faster inference (such as using model quantization
techniques or distilled models) can significantly reduce the
time it takes to generate summaries, especially for longer

38
 emails.
· 2.Handling Multilingual Text: Although the T5 model
is powerful, it is computationally intensive. Future versions
of the system can explore ways to reduce the model’s
size, such as through distillation or pruning, to ensure the
model is more lightweight while retaining its summarization
capabilities.

· Enhancing User Interaction

· 1.Customization of Summaries: Allowing users to cus-
tomize the summary generation process would add significant
value. Users could select summary lengths (short, medium,
long), the level of detail, or even the tone (formal, casual).
This can be achieved by further training the model on var-
ied types of summaries and incorporating user preferences
into the summarization pipeline.
· 2.Interactive Dashboard: The addition of an interactive
dashboard where users can track their summarization history,
adjust settings, and view summary statistics would enhance
the user experience. Integrating more advanced UI/UX
design principles can make the system more engaging.

· Integration with Other Platforms

· 1.Email Client Integration: One of the most valuable
improvements could be integrating this summarization sys-
tem directly into popular email clients like Gmail, Outlook,
or Yahoo Mail. By doing so, users could receive automatic
email summaries, reducing their email overload and helping
them focus on the most important communications.
· 2.API for Third-Party Services: Offering the summa-
rization system as a RESTful API would enable third-party
developers to integrate the summarizer into their applica-
tions. For example, it could be used in customer service
chatbots, news aggregation apps, or even enterprise docu-

Department of Information Technology 39

 ment management systems.
· Improving Model Performance
· 1.Fine-Tuning with Domain-Specific Data: The system
currently uses a general summarization dataset. To improve
its performance in specific industries, the model can be
fine-tuned using domain-specific datasets. For example, in-
tegrating datasets for legal contracts, medical records, or
customer support conversations can make the summariza-
tion system more specialized and effective in these areas.
· 2.Handling Multilingual Text: Currently, the model is
trained to summarize English text. Future improvements
can include extending its capabilities to support multiple
languages. This can be achieved by training or fine-tuning
multilingual versions of the T5 model, such as mBART
or mT5, to allow users to summarize emails in various
languages
· Exploring Advanced NLP Techniques
· 1.Incorporating Knowledge Graphs: To improve the
contextual relevance of the summaries, the system can in-
corporate knowledge graphs to understand the relationships
between different entities mentioned in the email. This
could lead to more coherent summaries that retain impor-
tant contextual information.
· 2.Multimodal Summarization: Another exciting future
direction could involve summarizing not only text but also
images, videos, or audio associated with emails. This would
require a multimodal model that can process both tex-
tual and visual data, generating summaries that integrate
information from multiple sources.
· Integration with Cloud and Edge Computing
· 1.Cloud Deployment:Deploying the summarization model
on the cloud can enhance accessibility, scalability, and se-

Department of Information Technology 40

 curity. Platforms like AWS, Google Cloud, and Microsoft
Azure offer powerful infrastructure for hosting AI models
and serving them to a large user base.

· 2.Edge Computing: For users who require offline sum-

marization capabilities, future iterations could explore edge
computing solutions, where the summarization model is de-
ployed directly on users’ devices, reducing reliance on inter-
net connectivity.

· User Feedback and Continuous Learning

· 1.Incorporating User Feedback: Allowing users to pro-

vide feedback on the summaries (e.g., rating summaries as
good or bad) would enable the system to learn from user
preferences and improve over time. A continuous learning
mechanism can be implemented to update the model with
new data and user insights.

· 2.Active Learning: Another approach for improving model

performance is active learning, where the system actively
selects the most informative examples to label and add to
the training dataset, allowing it to continuously adapt to
new data and user behavior.

Department of Information Technology 41

 REFERENCES
[1] Google Gemini AI Documentation:
Google Cloud. (2024). Google Gemini AI. Retrieved from
https://cloud.google.com
[2] React Documentation: React Team. (2024). React: A
JavaScript library for building user interfaces. Retrieved from
https://react.dev
[3] TypeScript Documentation: Microsoft. (2024). Type-
Script: JavaScript with syntax for types. Retrieved from
https://www.typescriptlang.org
[4] Tailwind CSS Documentation: Tailwind Labs. (2024). Tail-
wind CSS: Rapidly build modern websites without leaving your
HTML. Retrieved from https://tailwindcss.com
[5]Lucide React Documentation: Lucide. (2024). Lucide: Beau-
tiful customizable open-source icons for React. Retrieved from
https://lucide.dev
[6] dotenv Documentation: python-dotenv. (2024). Python-
dotenv: Read environment variables from a .env file. Retrieved
from https://pypi.org/project/python-dotenv
[7]Generative Models in AI: Vaswani, A., Shazeer, N., Parmar,
N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., Polo-
sukhin, I. (2017). Attention is all you need. In Proceedings of
NeurIPS.
[8]Natural Language Processing (NLP) Overview: Manning,
C.D., Schütze, H. (1999). Foundations of Statistical Natu-
ral Language Processing. MIT Press.

Department of Information Technology 42