While the robotics/embodiment is essential for phenomenology of artifacts, as it incurs too much technical complication, it is put off until relevant tools are readily available. The new plan also drops the emphasis on the recognition of Spelke's objects, which would require action during perception.
PurposeCreating a system that performs language acquisition (symbol grounding) with
Language acquisitionThe system will follow the human (infant) language acquisition process: it shall associate linguistic expressions with the shapes, colors, movements and relations of perceived objects, while creating adequate internal syntactic/semantic representations.
Core cognitive architectureThe core cognitive architecture shall have the following functions.
Upon designing and implementing the cognitive architecture, generic mechanisms shall be (re)used whenever possible.
- Time-series pattern learner/recognizer
motor commands andsensory input as their input
- Attention and situation assessment
on what will be learned
and which action will be taken.
- Cognitive model based on association
to memorize and recollect (temporal) generative patterns as associative sequences.
Linguistic competence will be realized with this model.
It contains backtracking mechanism based on the function of attention and situation assessment mentioned above.
- Episodic memory
Patterns (the representation of situations -- combinations of abstract patterns created by (non-supervised) learning) positively assessed by the attention and situation assessment will be memorized.
- Visual data processing
OpenCV, SimpleCV, etc.
- Learning module
BESOM, DeSTIN, k-means, SOM, SVN, HMM, SOINN, etc.
(To be used as plug-ins depending on the purpose)
- Publicly available cognitive architectures may be surveyed
e.g., OpenCog and LIDA
A Tentative Research Steps
Phase I: Survey on robot simulators (done)
Research on robot simulators such as SigVerse and V-rep and trial on attitude control.
Phase II: Survey on Spelke's Object recognition (done)
Proper recognition of Spelke's objects, having coherent, solid & inert bundle of features of a certain dimension that continues over time, would require optical flow processing associated with the action of the perceiver.
The following part doesn't have much change from the previous plan.
Phase III: Labeling
- Basic Ideas
- The system shall relate specific types of figures it looks at with linguistic labels.
- Figures get the system's attention by force (exterior instruction)
- Labels may be nominals representing shapes and adjectives representing features such as colors.
Types of objects may be learned in a supervised manner with labels or have been categorized by non-supervised learning.
- The system shall utter labels on recognizing types after learning association between the labels and types.
- The system shall recognize/utter the syntactic pattern 'adjective + noun'.
- Determining the recognition method & implementation
- Designing and implementing mechanism for handling syntactic structure.
- Incorporating episodic memory at this stage is to be considered.
- Labeling experiment
Phase IV: Relation Learning
- Basic Ideas
- The system shall learn labels for
- object locomotion such as moving up/down, right/left and closer/away
- orientational relations between objects such as above/below, right/left and short/thither
- Objects should be get the system's attention by force (programming) or by certain preprogrammed mechanism of attention (such as attention to moving objects).
- Designing & implementing the mechanism
Phase V: Linguistic Interaction
- Basic Ideas
- The system shall answer to questions using labels learned in Phase III & Phase IV.
- The system shall respond to requests on its behavior.
- The system shall utter clarification questions.
- Designing & implementing mechanism for linguistic interaction
Phase VI: Episodic memory
- Basic Ideas
- Episodes (situations) to be memorized are the appearance of objects and changes in relations among them.
- The memory of novel objects and situations is prioritized.
- Designing & implementing episodic memory and attentional mechanism
- Designing & implementing episodic recollection & linguistic report.
Phase VII: More complicated syntax
- Basic Idea
The system shall understand/utter linguistic expressions having nested phrase structure.
- Designing & implementing nested phrase structure.