<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="bajpai-hanawal-2024-dadee">
    <titleInfo>
        <title>DAdEE: Unsupervised Domain Adaptation in Early Exit PLMs</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Divya</namePart>
        <namePart type="given">Jyoti</namePart>
        <namePart type="family">Bajpai</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Manjesh</namePart>
        <namePart type="given">Kumar</namePart>
        <namePart type="family">Hanawal</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2024-11</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Findings of the Association for Computational Linguistics: EMNLP 2024</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Yaser</namePart>
            <namePart type="family">Al-Onaizan</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Mohit</namePart>
            <namePart type="family">Bansal</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Yun-Nung</namePart>
            <namePart type="family">Chen</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Miami, Florida, USA</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Pre-trained Language Models (PLMs) exhibit good accuracy and generalization ability across various tasks using self-supervision, but their large size results in high inference latency. Early Exit (EE) strategies handle the issue by allowing the samples to exit from classifiers attached to the intermediary layers, but they do not generalize well, as exit classifiers can be sensitive to domain changes. To address this, we propose Unsupervised Domain Adaptation in EE framework (DAdEE) that employs multi-level adaptation using knowledge distillation. DAdEE utilizes GAN-based adversarial adaptation at each layer to achieve domain-invariant representations, reducing the domain gap between the source and target domain across all layers. The attached exits not only speed up inference but also enhance domain adaptation by reducing catastrophic forgetting and mode collapse, making it more suitable for real-world scenarios. Experiments on tasks such as sentiment analysis, entailment classification, and natural language inference demonstrate that DAdEE consistently outperforms not only early exit methods but also various domain adaptation methods under domain shift scenarios. The anonymized source code is available at https://github.com/Div290/DAdEE.</abstract>
    <identifier type="citekey">bajpai-hanawal-2024-dadee</identifier>
    <location>
        <url>https://aclanthology.org/2024.findings-emnlp.371</url>
    </location>
    <part>
        <date>2024-11</date>
        <extent unit="page">
            <start>6389</start>
            <end>6400</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T DAdEE: Unsupervised Domain Adaptation in Early Exit PLMs
%A Bajpai, Divya Jyoti
%A Hanawal, Manjesh Kumar
%Y Al-Onaizan, Yaser
%Y Bansal, Mohit
%Y Chen, Yun-Nung
%S Findings of the Association for Computational Linguistics: EMNLP 2024
%D 2024
%8 November
%I Association for Computational Linguistics
%C Miami, Florida, USA
%F bajpai-hanawal-2024-dadee
%X Pre-trained Language Models (PLMs) exhibit good accuracy and generalization ability across various tasks using self-supervision, but their large size results in high inference latency. Early Exit (EE) strategies handle the issue by allowing the samples to exit from classifiers attached to the intermediary layers, but they do not generalize well, as exit classifiers can be sensitive to domain changes. To address this, we propose Unsupervised Domain Adaptation in EE framework (DAdEE) that employs multi-level adaptation using knowledge distillation. DAdEE utilizes GAN-based adversarial adaptation at each layer to achieve domain-invariant representations, reducing the domain gap between the source and target domain across all layers. The attached exits not only speed up inference but also enhance domain adaptation by reducing catastrophic forgetting and mode collapse, making it more suitable for real-world scenarios. Experiments on tasks such as sentiment analysis, entailment classification, and natural language inference demonstrate that DAdEE consistently outperforms not only early exit methods but also various domain adaptation methods under domain shift scenarios. The anonymized source code is available at https://github.com/Div290/DAdEE.
%U https://aclanthology.org/2024.findings-emnlp.371
%P 6389-6400

Markdown (Informal)

[DAdEE: Unsupervised Domain Adaptation in Early Exit PLMs](https://aclanthology.org/2024.findings-emnlp.371) (Bajpai & Hanawal, Findings 2024)

• DAdEE: Unsupervised Domain Adaptation in Early Exit PLMs (Bajpai & Hanawal, Findings 2024)

ACL

• Divya Jyoti Bajpai and Manjesh Kumar Hanawal. 2024. DAdEE: Unsupervised Domain Adaptation in Early Exit PLMs. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 6389–6400, Miami, Florida, USA. Association for Computational Linguistics.