Sunday, June 4, 2017

Research Update (2017-06)

I haven't written on my research for long time.  Last time, I wrote about making an agent learn verbs.  I did my research in the direction and wrote a paper to appear at the BICA 2017 conference.  The research differed in a few points with the previous plan:
  • The agent (LL) learns first- and second-person pronouns as well as verbs.
  • The learning environment does not use the robot simulator, which would take too much time.
For the first point, I noticed that the issue of personal pronouns cannot be avoided to deal with a human-language-like language. While a learning agent may learn them by observing other agents' language use, it may learn only by interacting with a sole caretaker as well.  My research pursued the second possibility, which had not been addressed in previous works.
In the research, the learner 'babbled' while speaking and acting, both of which had limited choices (it was assumed that the repertoires had been learnt beforehand).  The utterances and actions are reinforced by acknowledgment of the caretaker.  Some utterances are also reinforced when the caretaker takes expected actions.  While the internal representations denoting actions and agents are given as symbols, their relation to words were learned from interaction with the caretaker through a language game.  (I'll put the link to the paper when published.  Besides, the source code can be found on GitHub at .)
In this work, the language model for the two-word sentences were fixed; i.e., it consists of a subject and a verb in that order.  I'm thinking of making an agent learn language models, perhaps with LSTM, for the next research topic.

Wednesday, December 28, 2016

AGI in Japan, 2016

While this year's AGI-related topics in Japan seem fewer than previous years (see my articles for 2015 and 2014), new players such as AIX Research Center at the University of Electro-Communications and Araya Brain Imaging came into the scene.  A public fund was given to a five-year project Comparison and Fusion of Artificial Intelligence and Brain Science.
SIG-AGI, a special interest group of the Japanese Society for AI (JSAI), held sessions at the JSAI 2016 Convention, three workshops, and an event at the 30th Anniversary of JSAI (index in Japanese).
The Whole Brain Architecture Initiative (WBAI)(NPO) held a symposium, five seminars for the general public, the second annual hackathon, and a session at ICONIP.

Below, I write the development of WBAI's activities as an insider.
WBAI has a specific approach for AGI using brain-like connectomic architecture and off-the-shelf machine learning algorithms.  Last year, it was seeking collaborators on this approach world-wide, as it could not find enough researchers domestically.  Besides, it was also in need of software infrastructure such as basic cognitive architecture and learning environments.  In April, Dwango AI Lab., an affiliated private research laboratory, held a hackathon on a new learning environment (LIS or Life in Silico) for intelligent agents.  It was a great success with around 150 participants.  At the time, the situation around A(G)I was rapidly changing overseas.  One thing was the advent of OpenAI, an NPO aiming to counter AGI development by commercial superpowers such as Google.  In 2015, people in WBAI and SIG-AGI were talking with people in the AGI community overseas about establishing a similar entity.  The endeavor was supposedly quenched by the inauguration of OpenAI.  Another thing happening overseas was impressive advances of DeepMind.  While WBAI had a rather long-term plan for promoting researches with the WBA approach, there was a concern that DeepMind would realize AGI in a nearer future in a somewhat WBA way (some details in my memo "A Scenario in which DeepMind will realize AGI by 2018").  So WBAI veered.
With the success at the hackathon in April, WBAI set eyes on the potential of 'hobbyist' engineers/researchers and held another hackathon on Deep Predictive Coding Networks (originally proposed by David Cox's group) in July and created a new development community called SIG-WBA (the course of events is documented here).  WBAI is now more focused on popularizing evolving A(G)I technologies.  Problems?  Recently, research institutes such as Google, OpenAI, and GoodAI have released learning environments for intelligent agents.  That's a good thing (also for WBAI), but it requires human resources to check them out.  More importantly, WBAI is yet to develop basic connectomic architecture to be the core of their approach to attract researchers and engineers anywhere.  (The official version of WBAI's policy can be found here, which, of course, continues to evolve.  To keep you updated, follow WBAI on Twitter ...)

Monday, September 19, 2016

Toward Simple Language (Verb) Learning

I kind-of finished with the simulation environment (the previous post).
The LU (caretaker) now 'instructs' the LL (language learner) with commands such as 'stay,' 'turn,' 'come,' and 'go to the blue item' (in Interlingua).
"Luca, turn right."
When LL 'follows' the instruction, LU smiles and says "Good" to give a reward.

LU also 'declares' (thinks aloud) its own actions before carrying them out.
"I go to the blue one."

Now I guess I should go for language learning.
Here is a tentative plan:
(For the time being, I'll forget about LU's declaration mode, which requires the recognition of LU's behavior on LL's part.)
  1. The association of LU's words with LL's own actions
    As this should be done at a conceptual level, it requires:
    • Generation of action concepts
    • Generation of word concepts
    LL may carry out three kinds of learning:
    • Reinforcement learning of action under instruction
    • Association between word concepts and action concepts
    • Creating a language (syntactic) model linked with the concepts
  2. Bubbling
    LL will utter based on the language model.
    • If the utterance matches its action, LU repeats it and gives a reward.
      This reinforces the bubbling.
    • If not, LU describes LL's action after saying "No."
      This alters the conceptual association and language model.

Monday, August 1, 2016

New architecture

I altered the agent architecture.  The previous one used competing production-like rules for control.  The current one uses simpler perceive-reward-action cycles.  In case conflicts arise, they will be resolved by the action selector.  This would be more compatible with regular reinforcement learning architecture.  Besides the abolishment of competing rules, the rest is the same as the old design.
BTW, two agents are now split to the Language Learner and the Language User (caregiver).  LL chooses its action randomly and LU calls LL when LL is not looking back to LU.  LU's call is displayed as a small rectangular balloon on it, containing LL's current name "Luca."  Agents also 'smile' when they are 'looking into each other.'  These are part of the plan in my verb learning experiments.


The simulator was made with V-Rep and python.
The code is found on GitHub (Version 2 snapshot: 2016-08-01).

Monday, June 13, 2016

Carrying items

I made the agents to carry items to a given direction by pushing them.
This is a trick to be carried out in verb learning experiments.
I thought I'd use reinforcement learning to make them do the trick and tried to use PyBrain, but failed to use it without learning the cause.  I got back to the old-fashioned programming and fixed the trick while keenly feeling my lack of sense in geometry.  The agent first goes to the opposite side around the item against the carrying direction, start pushing, and adjusts the direction when the item veers.

The simulator was made with V-Rep and python.
The code is found on GitHub (Version 1 snapshot: 2016-06-13).

Friday, December 18, 2015

Thought on Architecture

The recent posts showed the agents under development in action.
Perhaps, it is the time to stop and think before continuing the development...

For the moment, the agents have very simple architecture, with a blackboard kind of internal communication system.  Namely, an agent has an input buffer and a state buffer, where 'codelets' can read from and write to, respectively.  As both buffers are python dictionaries (hash tables), they can contain any data, which are accessed by 'names/keys.'

Codelets are registered as 'rules,' which have a condition part and an action part.  In each execution cycle, the action of the codelets/rules whose conditions meet 'fires' and it may try to modify the content of the state buffer.  Each codelet has a score and if more than one codelet tries to modify a same part of the state buffer (with the same name/key), the value written by the codelet with the highest score (the winner) is chosen.

In the recent posts, I mentioned affect terms such as 'reward' and 'urge.'  In fact, they correspond to variables with the same names on the state buffer, and the agents 'smile' when they read a positive 'reward' in the buffer.

While the mechanism is quite simple, generic and usable, there are a few points to consider for future modifications:

  • Is it appropriate as a cognitive architecture?
    To make it look more like a respectable cognitive architecture, perhaps, the affect mechanism should have its own modules.
  • Is it biologically plausible?
    (The answer is, of course, No.)  To make it biologically plausible, the architecture should mimic the brain architecture.  Or, more simply, real brains are not supposed to use blackboard architecture.
  • Symbolic representation?
    The current system uses symbolic representation (python dictionaries) in internal communication.  Besides that the brain does not use such symbolic representation, using vector representation would be good when the system is to be controlled machine learning algorithms.

Tuesday, December 15, 2015

Spontaneous walk

After 'looking into each other' a while, the agents get bored (develop the urge to move) and start moving.  (The previous post showed the footage in which they follow each other, stop and smile when they meet.)

The simulator was made with V-Rep and python.
The code is found on GitHub.