Gijs Wijnholds

Gijs Wijnholds

Assistant Professor at Leiden University

Posts by Collection

portfolio

publications

Investigations into categorial grammar: Symmetric pregroup grammar and displacement calculus

Published in BSc Thesis, Utrecht University, 2011

In this thesis we investigate a symmetric extension of pregroup grammars, and the generative capacity of the Displacement Calculus. Read more

Recommended citation: Wijnholds, G. (2011). "Investigations into categorial grammar: Symmetric pregroup grammar and displacement calculus." BSc Thesis, Utrecht University. https://dspace.library.uu.nl/bitstream/handle/1874/207634/scriptie_final.pdf?sequence=2

Categorical Foundations for Extended Compositional Distributional Models of Meaning

Published in MSc Thesis, 2014

In the thesis we investigate a categorical compositional distributional model for the Lambek-Grishin Calculus, a two-sided extension of the Lambek Calculus. We also look at graphical languages for reasoning with diagrams for this extended model. Read more

Recommended citation: Wijnholds, G. (2014). "Categorical Foundations for Extended Compositional Distributional Models of Meaning." MSc Thesis, Institute for Logic, Language, and Computation. https://www.illc.uva.nl/Research/Publications/Reports/MoL-2014-22.text.pdf

Conversions between D and MCFG: Logical Characterizations of the Mildly Context-Sensitive Languages

Published in Computational Linguistics in the Netherlands Journal, 2014

This paper describes a characterisation of well-nested Multiple Context Free Languages by first order Displacement Calculus. Read more

Recommended citation: Wijnholds, G. J. (2014). "Conversions between D and MCFG: Logical Characterizations of the Mildly Context-Sensitive Languages." Computational Linguistics in the Netherlands Journal. 4. https://clinjournal.org/sites/clinjournal.org/files/10-Wijnholds-CLIN2014.pdf

Classical Copying versus Quantum Entanglement in Natural Language: The Case of VP-ellipsis

Published in Proceedings of the 2018 workshop on Compositional Approaches in Physics, NLP and Social Sciences, 2018

We develop and compare two compositional distributional models for embedding VP-elliptical phrases. Read more

Recommended citation: Wijnholds, G. and Sadrzadeh, M. (2018). "Classical Copying versus Quantum Entanglement in Natural Language: The Case of VP-ellipsis." CAPNS 2018 proceedings. http://eptcs.web.cse.unsw.edu.au/paper.cgi?CAPNS2018.8

A Proof-Theoretic Approach to Scope Ambiguity in Compositional Vector Space Models

Published in Journal of Language Modelling, 2019

Journal of Language Modelling. We combine a proof-theoretic model of scope ambiguities with a lexical treatment of quantifiers in a compositional distributional model. Read more

Recommended citation: Wijnholds, G. "A Proof-Theoretic Approach to Scope Ambiguity in Compositional Vector Space Models." Journal of Language Modelling, Vol 6, No 2. http://jlm.ipipan.waw.pl/index.php/JLM/article/view/232

A Type-Driven Vector Semantics for Ellipsis with Anaphora using Lambek Calculus with Limited Contraction

Published in LACompLing 2017, 2019

We develop a vector space semantics for verb phrase ellipsis with anaphora using type-driven compositional distributional semantics based on the Lambek calculus with limited contraction (LCC) of Jäger (2006). Read more

Recommended citation: Wijnholds, G. and Sadrzadeh, M. (2019). "A Type-Driven Vector Semantics for Ellipsis with Anaphora using Lambek Calculus with Limited Contraction." Journal of Logic, Language and Information, 2019. https://link.springer.com/article/10.1007/s10849-019-09293-4

Representation Learning for Type-Driven Composition

Published in CoNLL 2020, 2020

We implement a tensor-based skipgram architecture to learn verb matrix representations. These outperform previous methods in tensor-based representation learning and approach state of the art deep neural network models on the SICK dataset. Read more

Recommended citation: Wijnholds, G., and Sadrzadeh, M. and Clark, S. (2020). "Representation Learning for Type-Driven Composition." Proceedings of the 24th Conference on Computational Natural Language Learning. https://www.aclweb.org/anthology/2020.conll-1.24.pdf

Fuzzy Generalised Quantifiers for Natural Language in Categorical Compositional Distributional Semantics

Published in Mathematics, Logic, and Their Philosophies: Essays in Honour of Mohammad Ardeshir, Logic, Epistemology, and the Unity of Science, 2021

We develop the mathematics to deal with fuzzy generalized quantifiers in a compositional distributional model of meaning. Read more

Recommended citation: Dostal, M. and Sadrzadeh, M. and Wijnholds, G. (2020). "Fuzzy Generalised Quantifiers for Natural Language in Categorical Compositional Distributional Semantics." Mathematics, Logic, and Their Philosophies: Essays in Honour of Mohammad Ardeshir, Logic, Epistemology, and the Unity of Science. https://books.google.nl/books?hl=en&lr=&id=cYYbEAAAQBAJ&oi=fnd&pg=PA135&dq=info:2U-2e6HfwZQJ:scholar.google.com&ots=t-Jz96lQA5&sig=tiiz84KVxLNpSlowK81sB1tW8M8

SICK-NL: A Dataset for Dutch Natural Language Inference

Published in EACL 2021, 2021

We provide a semi-automatic translation of the SICK dataset into Dutch, providing the first Dutch Natural Language Inference task. Experiments suggest that the Dutch task is harder, and that state of the art language models do not fully capture syntactic invariance. Read more

Recommended citation: Wijnholds, G. and Moortgat, M. (2021). "SICK-NL: A Dataset for Dutch Natural Language Inference." Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics. https://aclanthology.org/2021.eacl-main.126/

Improving BERT Pretraining with Syntactic Supervision

Published in ArXiV, 2021

We train a BERT model from scratch for Dutch, while incorporating a supertagging objective to induce a syntactic bias. Initial experiments hint at improved or equal performance on a number of tasks, despite pretraining on a small amount of data. Read more

Recommended citation: Tziafas, G. and Kogkalidis, K. and Wijnholds, G. and Moortgat, M. (2021). "Improving BERT Pretraining with Syntactic Supervision." ArXiV preprint. https://arxiv.org/pdf/2104.10516.pdf

Anaphora and Ellipsis in Lambek Calculus with a Relevant Modality: Syntax and Semantics

Published in Journal of Cognitive Science, 2021

We use a DisCoCat vector space instance of semantics and show how one can interpret anaphora, ellipsis, and for the first time derive the sloppy vs strict vector readings of ambiguous anaphora with ellipsis cases. Read more

Recommended citation: McPheat, L. and and Wijnholds, G. and Sadrzadeh, M. and Correia, A. and Toumi, A. (2021). "Anaphora and Ellipsis in Lambek Calculus with a Relevant Modality: Syntax and Semantics." Journal of Cognitive Science. https://www.cl.cam.ac.uk/events/act2021/papers/ACT_2021_paper_42.pdf

Discontinuous Constituency and BERT: A Case Study of Dutch

Published in Findings of ACL 2022, 2022

We devise a test suite based on a mildly context-sensitive formalism, from which we derive grammars that capture the linguistic phenomena of control verb nesting and verb raising. The grammars, paired with a small lexicon, provide us with a large collection of naturalistic utterances, annotated with verb-subject pairings, that serve as the evaluation test bed for an attention-based span selection probe. Our results, backed by extensive analysis, suggest that the models investigated fail in the implicit acquisition of the dependencies examined. Read more

Recommended citation: Kogkalidis, K. and Wijnholds, G. (2022). "Discontinuous Constituency and BERT: A Case Study of Dutch." Findings of the Association for Computational Linguistics 2022. https://aclanthology.org/2022.findings-acl.298/

Noun Phrase and Verb Phrase Ellipsis in Dutch: Identifying Subject-Verb Dependencies with BERTje

Published in Computational Linguistics in the Netherlands 2022, 2022

We devise a test suite for evaluating Dutch language models' capacity for understanding noun and verb phrase ellipsis. The grammars, paired with a small lexicon, provide us with a large collection of naturalistic utterances, annotated with verb-subject pairings, that serve as the evaluation test bed for an attention-based span selection probe. Our results, backed by extensive analysis, suggest that the models investigated fail in the implicit acquisition of the dependencies examined. Read more

Recommended citation: Haagen, T. and Dona, L. and Bosscha, S. and Zamith, B. and Koetschruyter, R. and Wijnholds, G. (2022). "Noun Phrase and Verb Phrase Ellipsis in Dutch: Identifying Subject-Verb Dependencies with BERTje." Computational Linguistics in the Netherlands Journal Volume 12. https://www.clinjournal.org/clinj/article/view/147

Diamonds Are Forever: Theoretical and Empirical Support for a Dependency-Enhanced Type Logic

Published in Logic and Algorithms in Computational Linguistics 2021, 2023

Extended Lambek calculi enlarge the type language with adjoint pairs of unary modalities. In previous work, modalities have been used as licensors for controlled forms of restructuring, reordering and copying. Here, we study a complementary use of the modalities as dependency features coding for grammatical roles. The result is a multidimensional type logic simultaneously inducing dependency and function argument structure on the linguistic material. We discuss the new perspective on constituent structure suggested by the dependency-enhanced type logic, and we experimentally evaluate how well a neural language model like BERT can deal with the subtle interplay between logical and structural reasoning that this type logic gives rise to. Read more

Recommended citation: Moortgat, M. and Kogkalidis, K. and Wijnholds, G. (2023). "Diamonds Are Forever: Theoretical and Empirical Support for a Dependency-Enhanced Type Logic." Logic and Algorithms in Computational Linguistics 2021. https://link.springer.com/chapter/10.1007/978-3-031-21780-7_3

Assessing Monotonicity Reasoning in Dutch through Natural Language Inference

Published in Findings of EACL 2023, 2023

In this paper we investigate monotonicity reasoning in Dutch, through a novel Natural Language Inference dataset. Monotonicity reasoning shows to be highly challenging for Transformer-based language models in English and here, we corroborate those findings using a parallel Dutch dataset, obtained by translating the Monotonicity Entailment Dataset of Yanaka et al. (2019). After fine-tuning two Dutch language models BERTje and RobBERT on the Dutch NLI dataset SICK-NL, we find that performance severely drops on the monotonicity reasoning dataset, indicating poor generalization capacity of the models. We provide a detailed analysis of the test results by means of the linguistic annotations in the dataset. We find that models struggle with downward entailing contexts, and argue that this is due to a poor understanding of negation. Additionally, we find that the choice of monotonicity context affects model performance on conjunction and disjunction. We hope that this new resource paves the way for further research in generalization of neural reasoning models in Dutch, and contributes to the development of better language technology for Natural Language Inference, specifically for Dutch. Read more

Recommended citation: Wijnholds, G. (2023). "Assessing Monotonicity Reasoning in Dutch through Natural Language Inference." Findings of the Association for Computational Linguistics: EACL 2023. https://aclanthology.org/2023.findings-eacl.110/

Structural Ambiguity and its Disambiguation in Language Model Based Parsers: the Case of Dutch Clause Relativization

Published in ArXiV, 2023

This paper addresses structural ambiguity in Dutch relative clauses. By investigating the task of disambiguation by grounding, we study how the presence of a prior sentence can resolve relative clause ambiguities. We apply this method to two parsing architectures in an attempt to demystify the parsing and language model components of two present-day neural parsers. Results show that a neurosymbolic parser, based on proof nets, is more open to data bias correction than an approach based on universal dependencies, although both setups suffer from a comparable initial data bias. Read more

Recommended citation: Wijnholds, G. and Moortgat, M. (2023). "Structural Ambiguity and its Disambiguation in Language Model Based Parsers: the Case of Dutch Clause Relativization" ArXiV Preprint. https://arxiv.org/abs/2305.14917

Categorical Vector Space Semantics for Lambek Calculus with a Relevant Modality

Published in Compositionality Vol. 5 Issue 2, 2023

We develop a vector semantics based on the Lambek Calculus with a relevant modality, and give experimental result on a new dataset. Read more

Recommended citation: McPheat, L. and Sadrzadeh, M. and Wazni, H. and Wijnholds, G. (2020). "Categorical Vector Space Semantics for Lambek Calculus with a Relevant Modality." Compositionality Vol. 5, Issue 2. https://compositionality-journal.org/papers/compositionality-5-2/

talks

Compositional Distributional Semantics

Published:

I gave a guest lecture in Utrecht on compositional distributional semantics, followed by a lab with exercises using a concrete implementation of a compositional distributional model. Read more

Recommended citation: "Compositional Distributional Semantics".

Dependency Based Tensor Embeddings

Published:

I gave an invited talk about using dependency relations as contexts in a tensor-based embedding model. https://wollic2019.sites.uu.nl/compositionality-in-natural-languages/ Read more

Recommended citation: "Dependency Based Tensor Embeddings".

Neural Tensor Embeddings

Published:

I gave a guest lecture on current research on neural tensor embeddings in the Vector Space Models of Meaning course at ESSLLI 2019. https://sites.google.com/view/vsmom/home Read more

Recommended citation: "Neural Tensor Embeddings".

teaching

2010 2011 Teaching 14

Undergraduate, Utrecht University, 1900

An intermediate course on logical grammars and their semantics, from the basics of the Lambek Calculus to cutting-edge research topics. Read more

2010 2011 Teaching 15

Undergraduate, Utrecht University, 1900

Advanced course on techniques for language and speech technologies. Topics include formal languages (up to context-free languages), UNIX, hands on text-to-speech tutorials, Hidden Markov Models. Read more

2012 2013 Teaching 12

Undergraduate, University of Amsterdam, 1900

An introductory course on formal logic. Topics include propositional and first-order logic, natural deduction, sequent calculi, tableaux. Read more

2013 2014 Teaching 7

Undergraduate, University of Amsterdam, 1900

An introductory course to linear algebra, covering up to Singular Value Decomposition. Read more

2013 2014 Teaching 8

Undergraduate, University of Amsterdam, 1900

An intermediate course on logic for computational reasoning. Topics include propositional and first order resolution, dynamic epistemic logic, planning. Read more

2013 2014 Teaching 9

Undergraduate, University of Amsterdam, 1900

An introductory course to logic programming and various search techniques, in Prolog. Read more

2016 2017 Teaching 3

Undergraduate, Queen Mary University of London, 1900

Responsible tutoring and marking students' work for this introduction to mathematics for students of Electronic Engineering and Computer Systems students. Topics covered include complex numbers, vectors, sequences and series, functions and continuity, differentiation, and integration. Read more

2018 2019 Lecturing 1

Undergraduate, Queen Mary University of London, 1900

Responsible for organising and teaching this introduction to mathematics for students of Electronic Engineering and Computer Systems students. Topics covered include complex numbers, vectors, sequences and series, functions and continuity, differentiation, and integration. Read more

2018 2019 Teaching 4

Postgraduate, Queen Mary University of London, 1900

Responsible for running labs and marking for this capita selecta course on NLP for master students of various programmes. Topics covered include language modelling, formal grammar, dialogue modelling, sentiment analysis, classification and regression models, formal grammar, distributional semantics, formal semantics. Read more

2021 2022 Teaching 17

Postgraduate, Utrecht University, 1900

Advanced course on linear logic, grammar logics, formal semantics and distributional semantics. Read more