<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="same-etal-2023-models">
    <titleInfo>
        <title>Models of reference production: How do they withstand the test of time?</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Fahime</namePart>
        <namePart type="family">Same</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Guanyi</namePart>
        <namePart type="family">Chen</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Kees</namePart>
        <namePart type="family">van Deemter</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2023-09</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 16th International Natural Language Generation Conference</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">C</namePart>
            <namePart type="given">Maria</namePart>
            <namePart type="family">Keet</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Hung-Yi</namePart>
            <namePart type="family">Lee</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Sina</namePart>
            <namePart type="family">Zarrieß</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Prague, Czechia</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>In recent years, many NLP studies have focused solely on performance improvement. In this work, we focus on the linguistic and scientific aspects of NLP. We use the task of generating referring expressions in context (REG-in-context) as a case study and start our analysis from GREC, a comprehensive set of shared tasks in English that addressed this topic over a decade ago. We ask what the performance of models would be if we assessed them (1) on more realistic datasets, and (2) using more advanced methods. We test the models using different evaluation metrics and feature selection experiments. We conclude that GREC can no longer be regarded as offering a reliable assessment of models’ ability to mimic human reference production, because the results are highly impacted by the choice of corpus and evaluation metrics. Our results also suggest that pre-trained language models are less dependent on the choice of corpus than classic Machine Learning models, and therefore make more robust class predictions.</abstract>
    <identifier type="citekey">same-etal-2023-models</identifier>
    <identifier type="doi">10.18653/v1/2023.inlg-main.7</identifier>
    <location>
        <url>https://aclanthology.org/2023.inlg-main.7</url>
    </location>
    <part>
        <date>2023-09</date>
        <extent unit="page">
            <start>93</start>
            <end>105</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Models of reference production: How do they withstand the test of time?
%A Same, Fahime
%A Chen, Guanyi
%A van Deemter, Kees
%Y Keet, C. Maria
%Y Lee, Hung-Yi
%Y Zarrieß, Sina
%S Proceedings of the 16th International Natural Language Generation Conference
%D 2023
%8 September
%I Association for Computational Linguistics
%C Prague, Czechia
%F same-etal-2023-models
%X In recent years, many NLP studies have focused solely on performance improvement. In this work, we focus on the linguistic and scientific aspects of NLP. We use the task of generating referring expressions in context (REG-in-context) as a case study and start our analysis from GREC, a comprehensive set of shared tasks in English that addressed this topic over a decade ago. We ask what the performance of models would be if we assessed them (1) on more realistic datasets, and (2) using more advanced methods. We test the models using different evaluation metrics and feature selection experiments. We conclude that GREC can no longer be regarded as offering a reliable assessment of models’ ability to mimic human reference production, because the results are highly impacted by the choice of corpus and evaluation metrics. Our results also suggest that pre-trained language models are less dependent on the choice of corpus than classic Machine Learning models, and therefore make more robust class predictions.
%R 10.18653/v1/2023.inlg-main.7
%U https://aclanthology.org/2023.inlg-main.7
%U https://doi.org/10.18653/v1/2023.inlg-main.7
%P 93-105

Markdown (Informal)

[Models of reference production: How do they withstand the test of time?](https://aclanthology.org/2023.inlg-main.7) (Same et al., INLG-SIGDIAL 2023)

• Models of reference production: How do they withstand the test of time? (Same et al., INLG-SIGDIAL 2023)

ACL

• Fahime Same, Guanyi Chen, and Kees van Deemter. 2023. Models of reference production: How do they withstand the test of time?. In Proceedings of the 16th International Natural Language Generation Conference, pages 93–105, Prague, Czechia. Association for Computational Linguistics.