Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Discard Labels, add Resource Factory #725

Open
wants to merge 17 commits into
base: master
Choose a base branch
from
Open

Discard Labels, add Resource Factory #725

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
6ceb390
Add a property and feature allowing NER users to discard NER labels they
Jul 3, 2019
2de64ac
There was a bug in this test.
Jul 3, 2019
526f7fb
A score is not included with each constituent for named entity consti…
Jul 5, 2019
7d1d39f
Had to up the LBJava version.
Jul 5, 2019
63b7dd2
More updates, from 1.3.0 dependencies on LBJava to 1.3.2
Jul 5, 2019
371aa07
Some minor fixes, cleaned up the benchmark config files.
Jul 19, 2019
3441594
More fixes to config files.
Jul 19, 2019
6f292b2
A very small change to improve performance of brown clusters, improve…
Jul 22, 2019
77579ff
Caching paths on word proved quite effective, so I am checking that in.
Jul 23, 2019
08dc719
Added support for file type allowing the file type to be added to the
Jul 30, 2019
a9a52fc
Merge branch 'master' of https://github.com/cowchipkid/cogcomp-nlp
Jul 30, 2019
fd31974
I added a NERResourceManagerFactory and it's usage so resources can b…
Sep 13, 2019
67ed640
Fixed a performance issue adding constituents to a vew, allso some fi…
Sep 19, 2019
f526509
Caching paths was causing a race condition, and since I suspect addin…
Oct 25, 2019
c527e3c
Added methods to purge all memory for brown clusters and gazetteers.
Nov 12, 2019
67cf849
I had checked in some println with brown clusters erroneously, and th…
Dec 3, 2019
418452c
Model training cache files were not being deleted correctly, but now …
Dec 10, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions big-data-utils/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
<parent>
<artifactId>illinois-cogcomp-nlp</artifactId>
<groupId>edu.illinois.cs.cogcomp</groupId>
<version>4.0.15</version>
<version>4.0.19</version>
</parent>

<modelVersion>4.0.0</modelVersion>
Expand All @@ -23,7 +23,7 @@
<dependency>
<groupId>edu.illinois.cs.cogcomp</groupId>
<artifactId>illinois-core-utilities</artifactId>
<version>4.0.15</version>
<version>4.0.19</version>
</dependency>
<dependency>
<groupId>org.xeustechnologies.google-api</groupId>
Expand Down
10 changes: 5 additions & 5 deletions chunker/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
<parent>
<artifactId>illinois-cogcomp-nlp</artifactId>
<groupId>edu.illinois.cs.cogcomp</groupId>
<version>4.0.15</version>
<version>4.0.19</version>
</parent>

<modelVersion>4.0.0</modelVersion>
Expand All @@ -13,23 +13,23 @@
<dependency>
<groupId>edu.illinois.cs.cogcomp</groupId>
<artifactId>illinois-core-utilities</artifactId>
<version>4.0.15</version>
<version>4.0.19</version>
</dependency>

<dependency>
<groupId>edu.illinois.cs.cogcomp</groupId>
<artifactId>LBJava</artifactId>
<version>1.3.1</version>
<version>1.3.3</version>
</dependency>
<dependency>
<groupId>edu.illinois.cs.cogcomp</groupId>
<artifactId>LBJava-NLP-tools</artifactId>
<version>4.0.15</version>
<version>4.0.19</version>
</dependency>
<dependency>
<groupId>edu.illinois.cs.cogcomp</groupId>
<artifactId>illinois-pos</artifactId>
<version>4.0.15</version>
<version>4.0.19</version>
</dependency>
<dependency>
<groupId>edu.illinois.cs.cogcomp</groupId>
Expand Down
22 changes: 11 additions & 11 deletions commasrl/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
<parent>
<artifactId>illinois-cogcomp-nlp</artifactId>
<groupId>edu.illinois.cs.cogcomp</groupId>
<version>4.0.15</version>
<version>4.0.19</version>
</parent>
<modelVersion>4.0.0</modelVersion>

Expand All @@ -13,7 +13,7 @@
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
<lbjava-version>1.2.26</lbjava-version>
<lbjava-version>1.3.3</lbjava-version>
</properties>

<pluginRepositories>
Expand All @@ -35,48 +35,48 @@
<dependency>
<groupId>edu.illinois.cs.cogcomp</groupId>
<artifactId>illinois-core-utilities</artifactId>
<version>4.0.15</version>
<version>4.0.19</version>
<optional>true</optional>
</dependency>
<dependency>
<groupId>edu.illinois.cs.cogcomp</groupId>
<artifactId>illinois-curator</artifactId>
<version>4.0.15</version>
<version>4.0.19</version>
</dependency>
<dependency>
<groupId>edu.illinois.cs.cogcomp</groupId>
<artifactId>illinois-tokenizer</artifactId>
<version>4.0.15</version>
<version>4.0.19</version>
</dependency>
<dependency>
<groupId>edu.illinois.cs.cogcomp</groupId>
<artifactId>illinois-corpusreaders</artifactId>
<version>4.0.15</version>
<version>4.0.19</version>
</dependency>
<dependency>
<groupId>edu.illinois.cs.cogcomp</groupId>
<artifactId>illinois-inference</artifactId>
<version>4.0.15</version>
<version>4.0.19</version>
</dependency>
<dependency>
<groupId>edu.illinois.cs.cogcomp</groupId>
<artifactId>stanford_3.3.1</artifactId>
<version>4.0.15</version>
<version>4.0.19</version>
</dependency>
<dependency>
<groupId>edu.illinois.cs.cogcomp</groupId>
<artifactId>illinois-pos</artifactId>
<version>4.0.15</version>
<version>4.0.19</version>
</dependency>
<dependency>
<groupId>edu.illinois.cs.cogcomp</groupId>
<artifactId>illinois-ner</artifactId>
<version>4.0.15</version>
<version>4.0.19</version>
</dependency>
<dependency>
<groupId>edu.illinois.cs.cogcomp</groupId>
<artifactId>illinois-chunker</artifactId>
<version>4.0.15</version>
<version>4.0.19</version>
</dependency>
<dependency>
<groupId>edu.illinois.cs.cogcomp</groupId>
Expand Down
2 changes: 1 addition & 1 deletion core-utilities/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
<parent>
<artifactId>illinois-cogcomp-nlp</artifactId>
<groupId>edu.illinois.cs.cogcomp</groupId>
<version>4.0.15</version>
<version>4.0.19</version>
</parent>

<artifactId>illinois-core-utilities</artifactId>
Expand Down
8 changes: 5 additions & 3 deletions ...lities/src/main/java/edu/illinois/cs/cogcomp/core/datastructures/textannotation/View.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -137,7 +137,8 @@ private void removeAllTokenFromConstituentMapping(Constituent c) {

/**
* Convenience method for addConstituent(constituent, false)
* @param constituent
*
* @param constituent The new constituent to be added.
*/
public void addConstituent(Constituent constituent){
this.addConstituent(constituent, false);
Expand All @@ -148,9 +149,10 @@ public void addConstituent(Constituent constituent){
* Otherwise, we return the new constituent.
*
* @param constituent The new constituent to be added.
* @param force if true, add constituent even if it is a duplicate
*/
public void addConstituent(Constituent constituent, boolean force) {
if(!constituents.contains(constituent) || force) {
if(force || this.tokensToConstituents[constituent.getStartSpan()] == null || !constituents.contains(constituent)) {
constituents.add(constituent);

startSpan = Math.min(this.startSpan, constituent.getStartSpan());
Expand All @@ -161,7 +163,7 @@ public void addConstituent(Constituent constituent, boolean force) {
this.addTokenToConstituentMapping(token, constituent);
}
}
}else {
} else {
System.err.println("Warning (View.java): not adding duplicate Constituent: " + constituent + ", use addConstituent(c, true) to force add.");
}
}
Expand Down
14 changes: 9 additions & 5 deletions ...in/java/edu/illinois/cs/cogcomp/core/datastructures/vectors/ExceptionlessInputStream.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,8 +34,8 @@ public class ExceptionlessInputStream extends FilterInputStream {
private char[] chars = null;
/** The underlying data input stream. */
private DataInputStream dis;


/** if there is a zip stream, we must close it, closing the resulting stream does not close the file. */
private ZipFile zipfile = null;
/**
* Opens a buffered (and uncompressed) stream for reading from the specified file.
*
Expand Down Expand Up @@ -70,9 +70,9 @@ public static ExceptionlessInputStream openCompressedStream(String filename) {

try {
ZipFile zip = new ZipFile(filename);
eis =
new ExceptionlessInputStream(new BufferedInputStream(zip.getInputStream(zip
eis = new ExceptionlessInputStream(new BufferedInputStream(zip.getInputStream(zip
.getEntry(zipEntryName))));
eis.zipfile = zip;
} catch (Exception e) {
System.err.println("Can't open '" + filename + "' for input:");
e.printStackTrace();
Expand Down Expand Up @@ -160,7 +160,11 @@ private void handleException(Exception e) {
**/
public void close() {
try {
dis.close();
dis.close();
if (zipfile != null) {
zipfile.close();
zipfile = null;
}
} catch (Exception e) {
System.err.println("Can't close input stream:");
e.printStackTrace();
Expand Down
6 changes: 3 additions & 3 deletions corpusreaders/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
<parent>
<artifactId>illinois-cogcomp-nlp</artifactId>
<groupId>edu.illinois.cs.cogcomp</groupId>
<version>4.0.15</version>
<version>4.0.19</version>
</parent>

<artifactId>illinois-corpusreaders</artifactId>
Expand All @@ -15,12 +15,12 @@
<dependency>
<groupId>edu.illinois.cs.cogcomp</groupId>
<artifactId>illinois-core-utilities</artifactId>
<version>4.0.15</version>
<version>4.0.19</version>
</dependency>
<dependency>
<groupId>edu.illinois.cs.cogcomp</groupId>
<artifactId>illinois-tokenizer</artifactId>
<version>4.0.15</version>
<version>4.0.19</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
Expand Down
4 changes: 2 additions & 2 deletions curator/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
<parent>
<artifactId>illinois-cogcomp-nlp</artifactId>
<groupId>edu.illinois.cs.cogcomp</groupId>
<version>4.0.15</version>
<version>4.0.19</version>
</parent>

<artifactId>illinois-curator</artifactId>
Expand All @@ -16,7 +16,7 @@
<dependency>
<groupId>edu.illinois.cs.cogcomp</groupId>
<artifactId>illinois-core-utilities</artifactId>
<version>4.0.15</version>
<version>4.0.19</version>
</dependency>

<!-- Curator-related dependencies -->
Expand Down
6 changes: 3 additions & 3 deletions dataless-classifier/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
<parent>
<artifactId>illinois-cogcomp-nlp</artifactId>
<groupId>edu.illinois.cs.cogcomp</groupId>
<version>4.0.15</version>
<version>4.0.19</version>
</parent>

<modelVersion>4.0.0</modelVersion>
Expand All @@ -21,12 +21,12 @@
<dependency>
<groupId>edu.illinois.cs.cogcomp</groupId>
<artifactId>illinois-core-utilities</artifactId>
<version>4.0.15</version>
<version>4.0.19</version>
</dependency>
<dependency>
<groupId>edu.illinois.cs.cogcomp</groupId>
<artifactId>illinois-tokenizer</artifactId>
<version>4.0.15</version>
<version>4.0.19</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
Expand Down
12 changes: 6 additions & 6 deletions depparse/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
<parent>
<artifactId>illinois-cogcomp-nlp</artifactId>
<groupId>edu.illinois.cs.cogcomp</groupId>
<version>4.0.15</version>
<version>4.0.19</version>
</parent>

<artifactId>illinois-depparse</artifactId>
Expand All @@ -16,27 +16,27 @@
<dependency>
<groupId>edu.illinois.cs.cogcomp</groupId>
<artifactId>illinois-core-utilities</artifactId>
<version>4.0.15</version>
<version>4.0.19</version>
</dependency>
<dependency>
<groupId>edu.illinois.cs.cogcomp</groupId>
<artifactId>illinois-edison</artifactId>
<version>4.0.15</version>
<version>4.0.19</version>
</dependency>
<dependency>
<groupId>edu.illinois.cs.cogcomp</groupId>
<artifactId>illinois-lemmatizer</artifactId>
<version>4.0.15</version>
<version>4.0.19</version>
</dependency>
<dependency>
<groupId>edu.illinois.cs.cogcomp</groupId>
<artifactId>illinois-pos</artifactId>
<version>4.0.15</version>
<version>4.0.19</version>
</dependency>
<dependency>
<groupId>edu.illinois.cs.cogcomp</groupId>
<artifactId>illinois-chunker</artifactId>
<version>4.0.15</version>
<version>4.0.19</version>
</dependency>

<dependency>
Expand Down
10 changes: 5 additions & 5 deletions edison/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
<parent>
<artifactId>illinois-cogcomp-nlp</artifactId>
<groupId>edu.illinois.cs.cogcomp</groupId>
<version>4.0.15</version>
<version>4.0.19</version>
</parent>

<artifactId>illinois-edison</artifactId>
Expand All @@ -16,7 +16,7 @@
<dependency>
<groupId>edu.illinois.cs.cogcomp</groupId>
<artifactId>illinois-core-utilities</artifactId>
<version>4.0.15</version>
<version>4.0.19</version>
</dependency>
<!--<dependency>-->
<!--<groupId>edu.illinois.cs.cogcomp.resources</groupId>-->
Expand Down Expand Up @@ -80,13 +80,13 @@
<dependency>
<groupId>edu.illinois.cs.cogcomp</groupId>
<artifactId>illinois-corpusreaders</artifactId>
<version>4.0.15</version>
<version>4.0.19</version>
</dependency>
<!-- Used only in utilities.CreateTestTAResource -->
<dependency>
<groupId>edu.illinois.cs.cogcomp</groupId>
<artifactId>illinois-curator</artifactId>
<version>4.0.15</version>
<version>4.0.19</version>
<scope>test</scope>
</dependency>
<dependency>
Expand All @@ -98,7 +98,7 @@
<dependency>
<groupId>edu.illinois.cs.cogcomp</groupId>
<artifactId>LBJava</artifactId>
<version>1.3.0</version>
<version>1.3.3</version>
</dependency>
<!--<dependency>-->
<!--<groupId>edu.illinois.cs.cogcomp</groupId>-->
Expand Down
4 changes: 2 additions & 2 deletions external/clausie/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
<parent>
<artifactId>illinois-cogcomp-nlp</artifactId>
<groupId>edu.illinois.cs.cogcomp</groupId>
<version>4.0.15</version>
<version>4.0.19</version>
<relativePath>../../pom.xml</relativePath>
</parent>

Expand All @@ -24,7 +24,7 @@
<dependency>
<groupId>edu.illinois.cs.cogcomp</groupId>
<artifactId>external-commons</artifactId>
<version>4.0.15</version>
<version>4.0.19</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
Expand Down
6 changes: 3 additions & 3 deletions external/external-commons/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
<parent>
<artifactId>illinois-cogcomp-nlp</artifactId>
<groupId>edu.illinois.cs.cogcomp</groupId>
<version>4.0.15</version>
<version>4.0.19</version>
<relativePath>../../pom.xml</relativePath>
</parent>

Expand All @@ -16,12 +16,12 @@
<dependency>
<groupId>edu.illinois.cs.cogcomp</groupId>
<artifactId>illinois-core-utilities</artifactId>
<version>4.0.15</version>
<version>4.0.19</version>
</dependency>
<dependency>
<groupId>edu.illinois.cs.cogcomp</groupId>
<artifactId>illinois-tokenizer</artifactId>
<version>4.0.15</version>
<version>4.0.19</version>
</dependency>
<dependency>
<groupId>org.cogcomp</groupId>
Expand Down
Loading