Differences between revisions 30 and 33 (spanning 3 versions)
Revision 30 as of 2009-04-10 16:09:29
Size: 4371
Editor: AnnCopestake
Comment:
Revision 33 as of 2009-04-10 17:46:33
Size: 4674
Editor: AnnCopestake
Comment:
Deletions are marked like this. Additions are marked like this.
Line 3: Line 3:
The LexDB is useful for grammars with large lexicons. Rather than storing the lexicon in a .tdl file, which has to be loaded each time the lexicon is changed, the LexDB stores the lexical entries in a postgres database. The instructions for using the LexDB are somewhat sparse and it is not expected that novice LKB users will make use of the system. The best approach for users who want to make use of the LexDB for their own grammars is to start by getting a LexDB going with an existing grammar in order to become familiar with the set up. The initial choice that
has to be made is whether to use the Lex
DB in the 'single user' or 'multiple user' modes. In the multiple user mode, the revisions are stored in the database itself. The assumption is that the database is running on a server to which multiple users may have access. This mode assumes that the database is set up with its own id (something that system administrators may dislike because of potential security risks). In single user mode, the database user may be the actual user. Revisions are not stored in the database (the assumption is that Subversion or CVS will be used to manage changes).
The Lex DB is useful for grammars with large lexicons. Rather than storing the lexicon in a .tdl file, which has to be loaded each time the lexicon is changed, the Lex DB stores the lexical entries in a postgres database. The instructions for using the Lex DB are somewhat sparse and it is not expected that novice LKB users will make use of the system. The best approach for users who want to make use of the Lex DB for their own grammars is to start by working with an existing Lex DB in order to become familiar with the set up. The initial choice that
has to be made is whether to use the L
ex DB in the 'single user' or 'multiple user' modes. In the multiple user mode, the revisions are stored in the database itself. The assumption is that the database is running on a server to which multiple users may have access. This mode assumes that the database is set up with its own id (something that system administrators may dislike because of potential security risks). In single user mode, the database user may be the actual user. Revisions are not stored in the database (the assumption is that Subversion or CVS will be used to manage changes).
Line 6: Line 6:
* [http://www.cl.cam.ac.uk/~bmw20/Papers/ Papers] The ERG uses multi-user mode, but a single-user DB can be created by following the single user instructions below. Jacy uses a single user Lex DB.
Line 8: Line 8:
* LkbLexDbSingleUser for basic instructions for the SINGLE USER LexDB  * [http://www.cl.cam.ac.uk/~bmw20/Papers/ Papers]
Line 10: Line 10:
* LkbLexDbMultiUser for basic instructions for the MULTI USER LexDB  * LkbLexDbSingleUser for basic instructions for the SINGLE USER LexDB
Line 12: Line 12:
* JacyLexDb  * LkbLexDbMultiUser for basic instructions for the MULTI USER LexDB
Line 14: Line 14:
* LexDbEmacsInterface  * JacyLexDb
Line 16: Line 16:
* LexDbFieldMappings  * LexDbEmacsInterface
Line 18: Line 18:
* LexDbInternals  * LexDbFieldMappings
 
 
* LexDbInternals
Line 20: Line 22:
The remainder of this page contains miscellaneous information about use of the LexDb. The remainder of this page contains miscellaneous information about use of the Lex DB.
Line 24: Line 26:
The LexDB code requires that *features* includes :psql. This is now true by default for supported platforms. The Lex DB code requires that *features* includes :psql. This is now true by default for supported platforms.
Line 27: Line 29:

The filter is used to select subparts of a (multi-user) Lex DB. In the expanded LKB menu
Line 40: Line 45:
The filter determines the entries in the lexicon as seen by a particular user. Only revision entries matching the filter conditions in filter can form part of the lexicon (of these, the most recent revision is the one actually used). The filter determines the entries in the lexicon as seen by a particular user. Only revision entries matching the filter conditions in filter can form part of the lexicon (of these, the most recent revision is the one actually used).  The filter is not available in single-user mode.
Line 44: Line 49:
The LexDB may be dumped to text files which can then be uploaded to storage in CVS. The LexDB may be dumped to text files which can then be uploaded to storage in CVS or SVN.

LexDB Usage Instructions

The Lex DB is useful for grammars with large lexicons. Rather than storing the lexicon in a .tdl file, which has to be loaded each time the lexicon is changed, the Lex DB stores the lexical entries in a postgres database. The instructions for using the Lex DB are somewhat sparse and it is not expected that novice LKB users will make use of the system. The best approach for users who want to make use of the Lex DB for their own grammars is to start by working with an existing Lex DB in order to become familiar with the set up. The initial choice that has to be made is whether to use the Lex DB in the 'single user' or 'multiple user' modes. In the multiple user mode, the revisions are stored in the database itself. The assumption is that the database is running on a server to which multiple users may have access. This mode assumes that the database is set up with its own id (something that system administrators may dislike because of potential security risks). In single user mode, the database user may be the actual user. Revisions are not stored in the database (the assumption is that Subversion or CVS will be used to manage changes).

The ERG uses multi-user mode, but a single-user DB can be created by following the single user instructions below. Jacy uses a single user Lex DB.

The remainder of this page contains miscellaneous information about use of the Lex DB.

LKB Compilation

The Lex DB code requires that *features* includes :psql. This is now true by default for supported platforms.

HOW TO set the filter

The filter is used to select subparts of a (multi-user) Lex DB. In the expanded LKB menu

LexDB -> Set Filter

The filter specified is interpreted as an SQL WHERE clause. Examples:

     userid = 'danf'
     userid = 'danf' AND dialect = 'my_dialect'
     userid IN ('danf', 'aac')
     confidence > 0.5

(Note: the default is NULL. Change this to a non-NULL value to select a subset of the lexical database.)

The filter determines the entries in the lexicon as seen by a particular user. Only revision entries matching the filter conditions in filter can form part of the lexicon (of these, the most recent revision is the one actually used). The filter is not available in single-user mode.

HOW TO store LexDB in CVS

The LexDB may be dumped to text files which can then be uploaded to storage in CVS or SVN.

1. LexDB -> Dump

(This will dump public schema tables to text files -- eg. lexdb.rev, lexdb.rev_key, lexdb.dfn, lexdb.fld, lexdb.meta) BR(Note: the dump mechanism will also produce a .tdl file if *lexdb-dump-tdl* is set to t) BR(Note: the database dump files are tab-separated with null as \N)

2. Run the cvs commit command. E.g.

  • cvs commit ~/erg/lexdb.*

HOW TO retrieve LexDB from CVS

1. Run the cvs update command to retrieve the latest dump file. E.g.

  • cvs update ~/erg/lexdb.*

2. LexDB -> Load ('rev' entries)

These steps update the LexDB (public schema) to include all new revisions stored in a CVS dump file. The new entries will be copied to the table public.rev_new. Any changes made to your copy of the LexDB since the last update will be preserved.

HOW TO export LexDB to TDL file

LexDB -> Export (TDL file)

Dumps active LexDB entries (determined by filter) to .tdl file.

HOW TO import TDL entries from a file

To add a small number of new (revision) entries from a .tdl file: LexDB -> Import (TDL file). You will be queried to provide values for other certain non-grammar fields. The entries will go into the private rev table.

HOW TO commit entries to public rev

The LexDB consists of a single public schema and a set of private schemas, one per user. New (revision) entries are placed initially in your private schema. To commit (all) entries in your private schema to the public table: LexDB -> Commit private 'rev'

HOW TO list entries in private rev

From LKB: LexDB -> Display private 'rev'

or

from Emacs LexDB major mode: M-vs

HOW TO clear entries in private rev

LexDB -> Clear private 'rev

Troubleshooting

  • If M-x lexdb results in "no connection to LexDb", make sure that a working grammar is loaded.

LkbLexDb (last edited 2011-10-08 21:12:12 by localhost)

(The DELPH-IN infrastructure is hosted at the University of Oslo)