Skip to content

sacrebleu 2.6.0

Version history | Download JSON

  • PyPI https://pypi.org/project/sacrebleu/
    Repository https://github.com/mjpost/sacrebleu

  • py.typed

  • Coverage

    %%{init: {"pie": {"textPosition": 0.85}, "theme": "neutral", "themeVariables": {"pieStrokeWidth": "1px"}}}%%
pie title
    "Typed" : 268
    "Any" : 6
    "Untyped" : 261
    • 51.2% coverage
    • 50.1% coverage (strict)
    • 535 typable
      • 268 typed
      • 261 untyped
      • 6 Any

  • Typables

    %%{init: {"pie": {"textPosition": 0.85}, "theme": "neutral", "themeVariables": {"pieStrokeWidth": "1px"}}}%%
pie title
    "functions" : 148
    "classes" : 303
    "other" : 19
    • 38 functions (+0 overloads)
      • 110 parameters
    • 37 classes
      • 125 methods (+0 overloads)
        • 178 parameters
      • 0 properties
    • 30 modules
      • 19 attrs

Modules

Module Coverage Coverage (strict) Typables Ignores
sacrebleu.compat 86.3% 86.3% 51 0
sacrebleu.dataset 0.0% 0.0% 4 0
sacrebleu.dataset.base 17.1% 17.1% 41 0
sacrebleu.dataset.fake_sgml 0.0% 0.0% 14 0
sacrebleu.dataset.iwslt_xml 0.0% 0.0% 0 0
sacrebleu.dataset.plain_text 0.0% 0.0% 2 0
sacrebleu.dataset.tsv 0.0% 0.0% 5 0
sacrebleu.dataset.wmt_xml 0.0% 0.0% 12 0
sacrebleu.metrics 0.0% 0.0% 0 0
sacrebleu.metrics.base 71.9% 61.4% 57 0
sacrebleu.metrics.bleu 66.2% 66.2% 65 0
sacrebleu.metrics.chrf 68.2% 68.2% 44 0
sacrebleu.metrics.helpers 100.0% 100.0% 15 0
sacrebleu.metrics.lib_ter 88.9% 88.9% 18 0
sacrebleu.metrics.ter 64.5% 64.5% 31 0
sacrebleu.sacrebleu 25.0% 25.0% 4 1
sacrebleu.significance 46.7% 46.7% 30 0
sacrebleu.tokenizers.tokenizer_13a 0.0% 0.0% 4 0
sacrebleu.tokenizers.tokenizer_base 0.0% 0.0% 3 0
sacrebleu.tokenizers.tokenizer_char 0.0% 0.0% 4 0
sacrebleu.tokenizers.tokenizer_intl 50.0% 50.0% 4 0
sacrebleu.tokenizers.tokenizer_ja_mecab 0.0% 0.0% 5 0
sacrebleu.tokenizers.tokenizer_ko_mecab 0.0% 0.0% 5 0
sacrebleu.tokenizers.tokenizer_none 0.0% 0.0% 3 0
sacrebleu.tokenizers.tokenizer_re 0.0% 0.0% 4 0
sacrebleu.tokenizers.tokenizer_spm 0.0% 0.0% 11 0
sacrebleu.tokenizers.tokenizer_ter 75.0% 75.0% 8 0
sacrebleu.tokenizers.tokenizer_zh 0.0% 0.0% 6 0
sacrebleu.utils 36.7% 36.7% 79 0
sacrebleu.version 100.0% 100.0% 6 0

Incomplete Annotations

sacrebleu.compat (7 missing, 0 any)

Symbol Typable Typed Any
func sentence_bleu 8 7 0
func corpus_bleu 9 3 0

sacrebleu.dataset (4 missing, 0 any)

Symbol Typable Typed Any
attr DATASETS 1 0 0
attr SUBSETS 1 0 0
attr COUNTRIES 1 0 0
attr DOMAINS 1 0 0

sacrebleu.dataset.base (34 missing, 0 any)

Symbol Typable Typed Any
meth Dataset.init 8 5 0
meth Dataset.maybe_download 1 0 0
meth Dataset._clean 2 0 0
meth Dataset._get_tarball_filename 2 0 0
meth Dataset._get_txt_file_path 3 0 0
meth Dataset._get_langpair_metadata 2 0 0
meth Dataset.process_to_text 2 1 0
meth Dataset.fieldnames 2 1 0
meth Dataset.iter 2 0 0
meth Dataset.source 2 0 0
meth Dataset.references 2 0 0
meth Dataset.get_source_file 2 0 0
meth Dataset.get_reference_files 2 0 0
meth Dataset.get_files 2 0 0
attr Dataset.name 1 0 0
attr Dataset.data 1 0 0
attr Dataset.description 1 0 0
attr Dataset.citation 1 0 0
attr Dataset.md5 1 0 0
attr Dataset.langpairs 1 0 0
attr Dataset.kwargs 1 0 0

sacrebleu.dataset.fake_sgml (14 missing, 0 any)

Symbol Typable Typed Any
meth FakeSGMLDataset._convert_format 3 0 0
meth FakeSGMLDataset._convert_meta 4 0 0
meth FakeSGMLDataset.process_to_text 2 0 0
meth FakeSGMLDataset.fieldnames 2 0 0
meth WMTAdditionDataset._convert_format 3 0 0

sacrebleu.dataset.plain_text (2 missing, 0 any)

Symbol Typable Typed Any
meth PlainTextDataset.process_to_text 2 0 0

sacrebleu.dataset.tsv (5 missing, 0 any)

Symbol Typable Typed Any
meth TSVDataset._split_index_and_filename 3 0 0
meth TSVDataset.process_to_text 2 0 0

sacrebleu.dataset.wmt_xml (12 missing, 0 any)

Symbol Typable Typed Any
meth WMTXMLDataset._unwrap_wmt21_or_later 2 0 0
meth WMTXMLDataset._get_langpair_path 2 0 0
meth WMTXMLDataset.process_to_text 2 0 0
meth WMTXMLDataset._get_langpair_allowed_refs 2 0 0
meth WMTXMLDataset.get_reference_files 2 0 0
meth WMTXMLDataset.fieldnames 2 0 0

sacrebleu.metrics.base (16 missing, 6 any)

Symbol Typable Typed Any
meth Signature.init 2 1 0
meth Signature.update 3 2 1
meth Signature.str 1 0 0
meth Signature.repr 1 0 0
attr Signature.info 1 0 0
meth Metric.init 1 0 0
meth Metric._check_sentence_score_args 3 2 0
meth Metric._check_corpus_score_args 3 2 0
meth Metric._aggregate_and_compute 2 2 1
meth Metric._compute_score_from_stats 2 2 1
meth Metric._extract_corpus_statistics 3 3 1
meth Metric.sentence_score 3 3 1
meth Metric.corpus_score 4 4 1
attr Metric.n_bootstrap 1 0 0
attr Metric.seed 1 0 0
attr sacrelogger 1 0 0
meth Score.init 3 2 0
meth Score.estimate_ci 2 1 0
meth Score.repr 1 0 0
attr Score.name 1 0 0
attr Score.score 1 0 0

sacrebleu.metrics.bleu (22 missing, 0 any)

Symbol Typable Typed Any
meth BLEU.init 10 9 0
meth BLEU.compute_bleu 9 8 0
attr BLEU.TOKENIZERS 1 0 0
attr BLEU.trg_lang 1 0 0
attr BLEU.lowercase 1 0 0
attr BLEU.smooth_value 1 0 0
attr BLEU.smooth_method 1 0 0
attr BLEU.max_ngram_order 1 0 0
attr BLEU.effective_order 1 0 0
attr BLEU.tokenizer 1 0 0
attr BLEU.tokenizer_signature 1 0 0
attr sacrelogger 1 0 0
meth BLEUScore.init 8 7 0
attr BLEUScore.bp 1 0 0
attr BLEUScore.counts 1 0 0
attr BLEUScore.totals 1 0 0
attr BLEUScore.sys_len 1 0 0
attr BLEUScore.ref_len 1 0 0
attr BLEUScore.precisions 1 0 0
attr BLEUScore.prec_str 1 0 0
attr BLEUScore.ratio 1 0 0
meth BLEUSignature.init 2 1 0

sacrebleu.metrics.chrf (14 missing, 0 any)

Symbol Typable Typed Any
meth CHRF.init 8 7 0
attr CHRF._PUNCTS 1 0 0
attr CHRF.beta 1 0 0
attr CHRF.char_order 1 0 0
attr CHRF.word_order 1 0 0
attr CHRF.order 1 0 0
attr CHRF.lowercase 1 0 0
attr CHRF.whitespace 1 0 0
attr CHRF.eps_smoothing 1 0 0
meth CHRFScore.init 5 4 0
attr CHRFScore.beta 1 0 0
attr CHRFScore.char_order 1 0 0
attr CHRFScore.word_order 1 0 0
meth CHRFSignature.init 2 1 0

sacrebleu.metrics.lib_ter (2 missing, 0 any)

Symbol Typable Typed Any
meth BeamEditDistance.init 2 1 0
meth BeamEditDistance._add_cache 3 2 0

sacrebleu.metrics.ter (11 missing, 0 any)

Symbol Typable Typed Any
meth TER.init 6 5 0
attr TER.no_punct 1 0 0
attr TER.normalized 1 0 0
attr TER.asian_support 1 0 0
attr TER.case_sensitive 1 0 0
attr TER.tokenizer 1 0 0
attr TER.tokenizer_signature 1 0 0
meth TERSignature.init 2 1 0
meth TERScore.init 4 3 0
attr TERScore.num_edits 1 0 0
attr TERScore.ref_length 1 0 0

sacrebleu.sacrebleu (3 missing, 0 any)

Symbol Typable Typed Any
attr sacrelogger 1 0 0
func parse_args 1 0 0
func main 1 0 0

sacrebleu.significance (16 missing, 0 any)

Symbol Typable Typed Any
attr sacrelogger 1 0 0
meth Result.init 5 4 0
meth Result.repr 1 0 0
attr Result.score 1 0 0
attr Result.p_value 1 0 0
attr Result.mean 1 0 0
attr Result.ci 1 0 0
meth PairedTest.init 8 7 0
attr PairedTest.test_type 1 0 0
attr PairedTest.n_jobs 1 0 0
attr PairedTest.references 1 0 0
attr PairedTest.named_systems 1 0 0
attr PairedTest.n_ar_confidence 1 0 0
attr PairedTest.n_samples 1 0 0
attr PairedTest.n_systems 1 0 0
attr PairedTest.metrics 1 0 0

sacrebleu.tokenizers.tokenizer_13a (4 missing, 0 any)

Symbol Typable Typed Any
meth Tokenizer13a.signature 1 0 0
meth Tokenizer13a.init 1 0 0
meth Tokenizer13a.call 2 0 0

sacrebleu.tokenizers.tokenizer_base (3 missing, 0 any)

Symbol Typable Typed Any
meth BaseTokenizer.signature 1 0 0
meth BaseTokenizer.call 2 0 0

sacrebleu.tokenizers.tokenizer_char (4 missing, 0 any)

Symbol Typable Typed Any
meth TokenizerChar.signature 1 0 0
meth TokenizerChar.init 1 0 0
meth TokenizerChar.call 2 0 0

sacrebleu.tokenizers.tokenizer_intl (2 missing, 0 any)

Symbol Typable Typed Any
meth TokenizerV14International.signature 1 0 0
meth TokenizerV14International.init 1 0 0

sacrebleu.tokenizers.tokenizer_ja_mecab (5 missing, 0 any)

Symbol Typable Typed Any
meth TokenizerJaMecab.init 1 0 0
meth TokenizerJaMecab.call 2 0 0
meth TokenizerJaMecab.signature 1 0 0
attr TokenizerJaMecab.tagger 1 0 0

sacrebleu.tokenizers.tokenizer_ko_mecab (5 missing, 0 any)

Symbol Typable Typed Any
meth TokenizerKoMecab.init 1 0 0
meth TokenizerKoMecab.call 2 0 0
meth TokenizerKoMecab.signature 1 0 0
attr TokenizerKoMecab.tagger 1 0 0

sacrebleu.tokenizers.tokenizer_none (3 missing, 0 any)

Symbol Typable Typed Any
meth NoneTokenizer.signature 1 0 0
meth NoneTokenizer.call 2 0 0

sacrebleu.tokenizers.tokenizer_re (4 missing, 0 any)

Symbol Typable Typed Any
meth TokenizerRegexp.signature 1 0 0
meth TokenizerRegexp.init 1 0 0
meth TokenizerRegexp.call 2 0 0

sacrebleu.tokenizers.tokenizer_spm (11 missing, 0 any)

Symbol Typable Typed Any
attr sacrelogger 1 0 0
meth Flores200Tokenizer.init 1 0 0
meth TokenizerSPM.signature 1 0 0
meth TokenizerSPM.init 2 0 0
meth TokenizerSPM.call 2 0 0
attr TokenizerSPM.name 1 0 0
attr TokenizerSPM.sp 1 0 0
meth Flores101Tokenizer.init 1 0 0
meth spBLEU1KTokenizer.init 1 0 0

sacrebleu.tokenizers.tokenizer_ter (2 missing, 0 any)

Symbol Typable Typed Any
meth TercomTokenizer.init 5 4 0
meth TercomTokenizer.signature 1 0 0

sacrebleu.tokenizers.tokenizer_zh (6 missing, 0 any)

Symbol Typable Typed Any
meth TokenizerZh.signature 1 0 0
meth TokenizerZh.init 1 0 0
meth TokenizerZh._is_chinese_char 2 0 0
meth TokenizerZh.call 2 0 0

sacrebleu.utils (50 missing, 0 any)

Symbol Typable Typed Any
func smart_open 4 0 0
func download_test_set 3 0 0
attr SACREBLEU_DIR 1 0 0
attr sacrelogger 1 0 0
func get_available_subsets 3 1 0
func sanity_check_lengths 4 3 0
func filter_subset 6 0 0
func sum_of_lists 2 0 0
func print_test_set 6 0 0
func download_file 5 0 0
func get_md5sum 2 0 0
func get_available_origlangs 3 1 0
func args_to_dict 4 2 0
func print_results_table 4 3 0
func print_single_results 3 2 0
func extract_tarball 3 0 0
func print_subset_results 5 0 0
attr USERHOME 1 0 0
func get_files 3 1 0

Type-Ignore Comments

Flavor Count
type: ignore 1