|
<!DOCTYPE html>
|
|
<html lang="zh-CN">
|
<head>
|
<meta charset="utf-8" />
|
<meta name="viewport" content="width=device-width, initial-scale=1.0" /><meta name="generator" content="Docutils 0.18.1: http://docutils.sourceforge.net/" />
|
|
|
<!-- Licensed under the Apache 2.0 License -->
|
<link rel="stylesheet" type="text/css" href="_static/fonts/open-sans/stylesheet.css" />
|
<!-- Licensed under the SIL Open Font License -->
|
<link rel="stylesheet" type="text/css" href="_static/fonts/source-serif-pro/source-serif-pro.css" />
|
<link rel="stylesheet" type="text/css" href="_static/css/bootstrap.min.css" />
|
<link rel="stylesheet" type="text/css" href="_static/css/bootstrap-theme.min.css" />
|
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
|
<title>赛道设置与评估 — 多通道多方会议转录挑战2.0</title>
|
<link rel="stylesheet" type="text/css" href="_static/pygments.css" />
|
<link rel="stylesheet" type="text/css" href="_static/guzzle.css" />
|
<script data-url_root="./" id="documentation_options" src="_static/documentation_options.js"></script>
|
<script src="_static/jquery.js"></script>
|
<script src="_static/underscore.js"></script>
|
<script src="_static/_sphinx_javascript_frameworks_compat.js"></script>
|
<script src="_static/doctools.js"></script>
|
<script src="_static/sphinx_highlight.js"></script>
|
<script src="_static/translations.js"></script>
|
<script async="async" src="https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-mml-chtml.js"></script>
|
<link rel="index" title="索引" href="genindex.html" />
|
<link rel="search" title="搜索" href="search.html" />
|
<link rel="next" title="基线" href="%E5%9F%BA%E7%BA%BF.html" />
|
<link rel="prev" title="数据集" href="%E6%95%B0%E6%8D%AE%E9%9B%86.html" />
|
|
|
|
</head><body>
|
<div class="related" role="navigation" aria-label="related navigation">
|
<h3>导航</h3>
|
<ul>
|
<li class="right" style="margin-right: 10px">
|
<a href="genindex.html" title="总索引"
|
accesskey="I">索引</a></li>
|
<li class="right" >
|
<a href="%E5%9F%BA%E7%BA%BF.html" title="基线"
|
accesskey="N">下一页</a> |</li>
|
<li class="right" >
|
<a href="%E6%95%B0%E6%8D%AE%E9%9B%86.html" title="数据集"
|
accesskey="P">上一页</a> |</li>
|
<li class="nav-item nav-item-0"><a href="index.html">多通道多方会议转录挑战2.0</a> »</li>
|
<li class="nav-item nav-item-this"><a href="">赛道设置与评估</a></li>
|
</ul>
|
</div>
|
<div class="container-wrapper">
|
|
<div id="mobile-toggle">
|
<a href="#"><span class="glyphicon glyphicon-align-justify" aria-hidden="true"></span></a>
|
</div>
|
<div id="left-column">
|
<div class="sphinxsidebar"><a href="
|
index.html" class="text-logo">多通道多方会议转录挑战2.0</a>
|
<div class="sidebar-block">
|
<div class="sidebar-wrapper">
|
<div id="main-search">
|
<form class="form-inline" action="search.html" method="GET" role="form">
|
<div class="input-group">
|
<input name="q" type="text" class="form-control" placeholder="Search...">
|
</div>
|
<input type="hidden" name="check_keywords" value="yes" />
|
<input type="hidden" name="area" value="default" />
|
</form>
|
</div>
|
</div>
|
</div>
|
<div class="sidebar-block">
|
<div class="sidebar-toc">
|
|
|
<p class="caption" role="heading"><span class="caption-text">目录:</span></p>
|
<ul class="current">
|
<li class="toctree-l1"><a class="reference internal" href="%E7%AE%80%E4%BB%8B.html">简介</a><ul>
|
<li class="toctree-l2"><a class="reference internal" href="%E7%AE%80%E4%BB%8B.html#id2">竞赛介绍</a></li>
|
<li class="toctree-l2"><a class="reference internal" href="%E7%AE%80%E4%BB%8B.html#aoe">时间安排(AOE时间)</a></li>
|
<li class="toctree-l2"><a class="reference internal" href="%E7%AE%80%E4%BB%8B.html#id3">竞赛报名</a></li>
|
</ul>
|
</li>
|
<li class="toctree-l1"><a class="reference internal" href="%E6%95%B0%E6%8D%AE%E9%9B%86.html">数据集</a><ul>
|
<li class="toctree-l2"><a class="reference internal" href="%E6%95%B0%E6%8D%AE%E9%9B%86.html#id2">数据集概述</a></li>
|
<li class="toctree-l2"><a class="reference internal" href="%E6%95%B0%E6%8D%AE%E9%9B%86.html#alimeeting">Alimeeting数据集介绍</a></li>
|
<li class="toctree-l2"><a class="reference internal" href="%E6%95%B0%E6%8D%AE%E9%9B%86.html#id3">获取数据</a></li>
|
</ul>
|
</li>
|
<li class="toctree-l1 current"><a class="current reference internal" href="#">赛道设置与评估</a><ul>
|
<li class="toctree-l2"><a class="reference internal" href="#id2">说话人相关的语音识别</a></li>
|
<li class="toctree-l2"><a class="reference internal" href="#id3">评估方法</a></li>
|
<li class="toctree-l2"><a class="reference internal" href="#id4">子赛道设置</a></li>
|
</ul>
|
</li>
|
<li class="toctree-l1"><a class="reference internal" href="%E5%9F%BA%E7%BA%BF.html">基线</a><ul>
|
<li class="toctree-l2"><a class="reference internal" href="%E5%9F%BA%E7%BA%BF.html#id2">基线概述</a></li>
|
<li class="toctree-l2"><a class="reference internal" href="%E5%9F%BA%E7%BA%BF.html#id3">快速开始</a></li>
|
<li class="toctree-l2"><a class="reference internal" href="%E5%9F%BA%E7%BA%BF.html#id4">基线结果</a></li>
|
</ul>
|
</li>
|
<li class="toctree-l1"><a class="reference internal" href="%E8%A7%84%E5%88%99.html">竞赛规则</a></li>
|
<li class="toctree-l1"><a class="reference internal" href="%E7%BB%84%E5%A7%94%E4%BC%9A.html">组委会</a></li>
|
<li class="toctree-l1"><a class="reference internal" href="%E8%81%94%E7%B3%BB%E6%96%B9%E5%BC%8F.html">联系方式</a></li>
|
</ul>
|
|
|
</div>
|
</div>
|
|
</div>
|
</div>
|
<div id="right-column">
|
|
<div role="navigation" aria-label="breadcrumbs navigation">
|
<ol class="breadcrumb">
|
<li><a href="index.html">Docs</a></li>
|
|
<li>赛道设置与评估</li>
|
</ol>
|
</div>
|
|
<div class="document clearer body">
|
|
<section id="id1">
|
<h1>赛道设置与评估<a class="headerlink" href="#id1" title="此标题的永久链接">¶</a></h1>
|
<section id="id2">
|
<h2>说话人相关的语音识别<a class="headerlink" href="#id2" title="此标题的永久链接">¶</a></h2>
|
<p>说话人相关的ASR任务需要从重叠的语音中识别每个说话人的语音，并为识别内容分配一个说话人标签。图2展示了说话人相关语音识别任务和多说话人语音识别任务的主要区别。在本次竞赛中AliMeeting、Aishell4和Cn-Celeb数据集可作为受限数据源。在M2MeT挑战赛中使用的AliMeeting数据集包含训练、评估和测试集，在M2MeT2.0可以在训练和评估中使用。此外，一个包含约10小时会议数据的新的Test-2023集将根据赛程安排发布并用于挑战赛的评分和排名。值得注意的是，对于Test-2023测试集，主办方将不再提供耳机的近场音频、转录以及真实时间戳。而是提供可以通过一个简单的VAD模型得到的包含多个说话人的片段。</p>
|
<p><img alt="task difference" src="_images/task_diff.png" /></p>
|
</section>
|
<section id="id3">
|
<h2>评估方法<a class="headerlink" href="#id3" title="此标题的永久链接">¶</a></h2>
|
<p>使用串联最优排序字符错误率（cpCER）指标来评估说话人相关的ASR系统的准确性。cpCER的计算包括三个步骤。首先，将一场会议中每个说话人的参考和假设转录按时间顺序串联起来。其次，计算真实标签和预测输出之间的字符错误率（CER），并对所有可能的说话人排列组合重复这一过程。最后，选择CER最低的排列组合作为该时段的cpCER。CER是通过将ASR输出转化为参考抄本所需的插入（Ins）、替换（Sub）和删除（Del）的字符总数除以参考抄本的字符总数得到的。具体来说，CER的计算方法是：</p>
|
<div class="math notranslate nohighlight">
|
\[ \text{CER} = \frac {\mathcal N_{\text{Ins}} + \mathcal N_{\text{Sub}} + \mathcal N_{\text{Del}} }{\mathcal N_{\text{Total}}} \times 100\%, \]</div>
|
<p>其中 <span class="math notranslate nohighlight">\(\mathcal N_{\text{Ins}}\)</span> , <span class="math notranslate nohighlight">\(\mathcal N_{\text{Sub}}\)</span> , <span class="math notranslate nohighlight">\(\mathcal N_{\text{Del}}\)</span> 是三种错误的字符数, <span class="math notranslate nohighlight">\(\mathcal N_{\text{Total}}\)</span> 是字符总数.</p>
|
</section>
|
<section id="id4">
|
<h2>子赛道设置<a class="headerlink" href="#id4" title="此标题的永久链接">¶</a></h2>
|
<section id="id5">
|
<h3>子赛道一 (限定训练数据):<a class="headerlink" href="#id5" title="此标题的永久链接">¶</a></h3>
|
<p>参赛者在系统构建过程中仅能使用AliMeeting、AISHELL-4和CN-Celeb，严禁使用额外数据。参赛者可以任何第三方开源的预训练模型，如<a class="reference external" href="https://huggingface.co/models">Hugging Face</a>以及<a class="reference external" href="https://www.modelscope.cn/models">ModelScope</a>上提供的模型。参赛者需要在最终的系统描述文档中详细列出使用的预训练模型名称以及链接。</p>
|
</section>
|
<section id="id6">
|
<h3>子赛道二 (开放训练数据):<a class="headerlink" href="#id6" title="此标题的永久链接">¶</a></h3>
|
<p>除了限定数据外，参与者可以使用任何公开可用、私人录制和模拟仿真的数据集。但是，参与者必须清楚地列出使用的数据。同样，参赛者也可以使用任何第三方开源的预训练模型，但必须在最后的系统描述文件中明确的列出所使用的数据和模型链接，如果使用模拟仿真数据，请详细描述数据模拟的方案。</p>
|
</section>
|
</section>
|
</section>
|
|
|
</div>
|
|
<div class="footer-relations">
|
|
<div class="pull-left">
|
<a class="btn btn-default" href="%E6%95%B0%E6%8D%AE%E9%9B%86.html" title="上一章 (use the left arrow)">数据集</a>
|
</div>
|
|
<div class="pull-right">
|
<a class="btn btn-default" href="%E5%9F%BA%E7%BA%BF.html" title="下一章 (use the right arrow)">基线</a>
|
</div>
|
</div>
|
<div class="clearer"></div>
|
|
</div>
|
<div class="clearfix"></div>
|
</div>
|
<div class="related" role="navigation" aria-label="related navigation">
|
<h3>导航</h3>
|
<ul>
|
<li class="right" style="margin-right: 10px">
|
<a href="genindex.html" title="总索引"
|
>索引</a></li>
|
<li class="right" >
|
<a href="%E5%9F%BA%E7%BA%BF.html" title="基线"
|
>下一页</a> |</li>
|
<li class="right" >
|
<a href="%E6%95%B0%E6%8D%AE%E9%9B%86.html" title="数据集"
|
>上一页</a> |</li>
|
<li class="nav-item nav-item-0"><a href="index.html">多通道多方会议转录挑战2.0</a> »</li>
|
<li class="nav-item nav-item-this"><a href="">赛道设置与评估</a></li>
|
</ul>
|
</div>
|
<script type="text/javascript">
|
$("#mobile-toggle a").click(function () {
|
$("#left-column").toggle();
|
});
|
</script>
|
<script type="text/javascript" src="_static/js/bootstrap.js"></script>
|
<div class="footer">
|
© Copyright 2023, Speech Lab, Alibaba Group; ASLP Group, Northwestern Polytechnical University. Created using <a href="http://sphinx.pocoo.org/">Sphinx</a>.
|
</div>
|
</body>
|
</html>
|
