Кредо исследователя или как прожить в науке

Сегодня был на очень интересной лекции по машинному переводу, где
в неофициальной части докладчик (Matti Kääriäinen) рассмотрел проблемы академической
атмосферы. Главной проблемой он считает то, что основная масса
исследователей ориентируется на публикацию статей с целью публикации
статей. Он также привёл этапы аккуратного на его взгляд распорядка:

1) поставь научную цель и представь, что она решена - что тогда? Как
изменится мир? Если дальше, чем решение задачи, продумать не можешь,
выбери другую задачу;
2) keep the pace: делай даже маленькое сегодня (если нет веских причин
отложить на завтра);
3) публикуй свои достижения в любом виде: статьи, исходный код и т. д.

Также нужно убрать ненужные риски с плеч исследователя. Если,
например, одна группа исследует геном и полагает, что ген интеллекта
находится в хромосоме 7, а вторая - в хромосоме 9, то полугодичная
работа обеих групп с результатом "ген в хромосоме 9" не должен
означать конец карьеры участников первой группы. Эта ответственность
должна ложиться на профессора как главу исследовательской группы.

А что на практике: один исследователь получил контракт на 5 лет с
университетом. Полгода прошли активно, а после этого он стал постоянно
думать: а что через 4,5 года? И так очень многие: нужно жить, и люди
думают об этом очень много, стараясь выполнить все формальные
требования для продления контракта, забывая о главной цели.

Есть над чем подумать.

IR workshop @ AM-CP

has hosted another talk of mine. This time I have brought in more details on natural language functional theory and computer semantics. The audience has been rather wide as well: 10-15 people. After diving into computer semantics we've turned to Machine Translation topic, briefly covering statistical and classical approaches. It has been also important to see how the students follow the talk as well as keep them laughing from time to time to break the ice wall.

IR workshop @ AM-CP

Computer Semantics And Machine Translation

View more presentations from dmitrykan.

CPS'09

For those of you, who silently keep track of what's going on, my following message: I have made a conference talk last week at CPS'09 (Control Processes and Stability). The talk is in Russian. In brief: the article proposes an automatic method for creating a semantic translational dictionary ru->en. The method is based on statistical approaches and implemented with the use of GIZA++. Comparison of an experimental machine translation system based on the dictionary with the SMT system based on Moses is provided.

Automatic Build Of Semantic Translational Dictionary

View more presentations from dmitrykan.

autoflush feature in ActivePerl

I tend to post here only those receipts that have practically worked for me.

There have been another feature I couldn't make working under win32 for ActivePerl. It is the autoflush, i.e. auto flushing the buffer content to a physical file.

Spied here. The solution is to make the filehandle "hot":

select((select(FILEHANDLE), $| = 1)[0]); # make the handle hot

Analogy of a face search in a human brain

While I was preparing my breakfast a thought of a face expression flashed accross my mind. The small picture was so bright,
that I immediately started to search the owner in my memory. This is the exact verb, that I should use
here: to search. The search took just about two seconds (lucky me) and I started to analyze
how did it happen: what algorithm did I potentially use, what was the complexity. As I met the owner of the
face about a week ago I believe the search took place in cache and looked a linear search on one hand.
On the other hand I distinctly realized while this was happenning that I clearly was skipping some
faces without further detailed analysis (i.e. calculating some metrics for determining relevance level).
One important observation is that I didn't draw a clear line in a search space based on gender: rather it was a quick search (with no reference
to the existing algorithm, but an allegory) with very fast process of prunning the search space.


It might turn out, that existing search algorithms are not that far from what happens inside a human brain.


mood: thoughtful :)

podcasting on Google

I've decided to make the first podcast with a certain purpose: there is a podcast contest by budam on rpod.ru. The general idea is to create a podcast, speaking entirely in English which must not be your native language.

The podcast is on how I have had an interview with Google in Trondheim (pity, pity, recently closed office) and - Google be safe - does not disclose a single task from the NDA signed tasks list I got that long day. It actually was a long day - with 5 interviews and one lunch. I bet I have drunk a 5 litres bottle of water - when you are speaking up for 45 minutes per interview your throat gets easily parched.

Microsoft fights itself

No, no, no. It is not any kind of political post related to the company's current state. It is not a holy war type of post either. I have just got the below interesting message from which I concluded the topic.



I'll translate the key phrases as well:

Prevention of data execution - Microsoft Windows

For defending the computer this program was shut down by the system

Name: Windows®installer
Publisher: Microsoft Corporation

Локальная гравитация

Представьте, что вы отправились покупать вашу любимую пиццу в местной пиццерии. Предположим, вы не просто пошли туда, а поехали на велосипеде (да, захватите ещё рюкзак, ниже я объясню, зачем). Когда пицца, горячая и ароматная, запакованная в картонную коробку, у вас в руках, вы раздумываете, а как бы добраться теперь до дома с комфортом.

Если пиццу держать на руле, сохраняя её горизонтальное положение, будет сложно маневрировать. Особенно, если дорога бугристая ото льда. Можно пиццу привязать к багажнику, но это будет вызывать неудобство: нужно всякий раз оборачиваться и смотреть, на мести ли пицца.

Тогда можно положить пиццу в рюкзак. Стоп! Как же можно класть пиццу в рюкзак, она ведь соберёт все свои компоненты внизу рюкзака. Можно! Можно положить пиццу в рюкзак с с локальной гравитацией, вектор действия которой вы можете задать самостоятельно. Таким образом, гравитация для пиццы в рюкзаке будет направлена соосно вашему движению на велосипеде, но в противоположную сторону.

Осталось подумать о том, как достичь следующего:

1) локальная гравитация должна быть именно локальной, то есть не действовать, к примеру, на вашу шапку на голове;

2) локальная гравитация не должна конкурировать с естественной гравитацией Земли, в идеальном случае они должны существовать параллельно друг к другу, без каких бы то ни было интерференций;

3) можно произвольно менять вектор локальной гравитации.

Приятного аппетита! :)

Mr. Lecturer

Last week I have given lectures on Machine Translation (in Russian) in my home Saint-Petersburg State University.

Introduction To Machine Translation

View more presentations from guestae31e.

Introduction To Machine Translation (continuation)

View more presentations from guestae31e.

I let my students know beforehand that the course is experimental and they happen to be the pioneers whom the course is going to be tested on.

After the 3,5 h lecture I have asked them, how do they feel about the experiment.

The answer was: The course was interesting. In case it would have been uninteresting, they would sleep. I believe it is the best compliment, especially taking into account the fact, that a human concentrates only first 40 (15?) minutes.

Unicode in Perl

Sometimes it feels that perl's power in string manipulations comes at a cost of its synthax being awkward.

When you open a file for reading without caring in what encoding is its contents, you do:

open FILE, "<".$filename or die $!;

But if you do care of an encoding you should open the file using the following instruction:

open ENC_FILE, "<:encoding(cp1251)", $enc_filename or die $!;

Now the key point is in comma following the encoding instruction. If you put there "." instead (which I believe does the concatenation of stream direction sign "<" and the filename), the file fails to open.

Another important addition is: if you know in advance in which encoding the file contents is represented, specify it using the above encoding instruction. Doing this you get all the string data to be in internal perl's representation which is by default utf8.

I feel like on top of the world (c)

.. when I manage to make an unhadled exception in compiler / interpreter. This time it has happened with Perl Command Line Interpreter:

Perl: file or directory

To check this, the prescription says:

if (-d $file)
{
   print $file." is a directory\n";
} else {
   print $file." is a file\n";
}

When this is used in pair with IO:Dir, which helps you to enumerate contents of a given directory, one non-obvious step should not be forgotten:

tie %dir, 'IO::Dir', $dir;
foreach my $entry(keys %dir) {
   next if ($entry eq '.' or $entry eq '..');
   # important part is here: concatenation with the full path
   if (-d $dir."/".$entry)
   {
      print $entry." is a directory\n";
   }
}

Simple Perl modules

Making it already a rool to post technical details for which I have spent more than 20 minutes, I decided to post as well this.

keywords: How to write perl modules
PageRank: http://www.netalive.org/tinkering/serious-perl/

Answer: it's simple!

Create file StringManip.pm in Lib/ directory where you like with the following contents:

package Lib::StringManip;

use strict;

use base 'Exporter';
our @EXPORT = ('trim');

# Perl trim function to remove whitespace from the start and end of the string
sub trim
{
 my $string = shift;
 $string =~ s/^\s+//;
 $string =~ s/\s+$//;
 return $string;
}

1;

Add the full path to Lib/StringManip.pm to PERL5LIB environment variable. In my case (win32) it is: PERL5LIB=%PERL5LIB%;D:\Programming\Perl

i.e. inside D:\Programming\Perl I have Lib/StringManip.pm. In Linux/Unix: export PERL5LIB=some_path/Lib/StringManip.pm.

Usage snippet:

#!perl -w

use strict;
use Lib::StringManip;

print trim("  trim me!  ");

Natural Langauge Processing and preparation of a human brain

Having read a number of articles dealing with natural language processing (NLP), cognition and linguistics, like Beyond Zipf's law: Modeling the structure of human language, I come to a conclusion, that NLP in essence is one of the most accurate and non-intruding ways to understand how human brain works.

Compare NLP, for example to neuropsychology.

Moscow journey video via Animoto

That is very easy to make a video with Animoto and brings some freshness into your world.

How do I load an org.w3c.dom.Document from XML in a string?

The entire header of this message and it's contents were copied from http://stackoverflow.com/questions/33262/how-do-i-load-an-orgw3cdomdocument-from-xml-in-a-string with one aim: to give higher rank for the page in Google PageRank.

Good reason for this is that good things should be replicated.




I have a complete XML document in a string and would like a Document object. Google turns up all sorts of garbage. What is the simplest solution? (In Java 1.5)

Solution Thanks to Matt McMinn, I have settled on this implementation. It has the right level of input flexibility and exception granularity for me. (It's good to know if the error came from malformed XML - SAXException - or just bad IO - IOException.)

public static org.w3c.dom.Document loadXMLFrom(String xml)
    throws org.xml.sax.SAXException, java.io.IOException {
    return loadXMLFrom(new java.io.ByteArrayInputStream(xml.getBytes()));
}

public static org.w3c.dom.Document loadXMLFrom(java.io.InputStream is) 
    throws org.xml.sax.SAXException, java.io.IOException {
    javax.xml.parsers.DocumentBuilderFactory factory =
        javax.xml.parsers.DocumentBuilderFactory.newInstance();
    factory.setNamespaceAware(true);
    javax.xml.parsers.DocumentBuilder builder = null;
    try {
        builder = factory.newDocumentBuilder();
    }
    catch (javax.xml.parsers.ParserConfigurationException ex) {
    }  
    org.w3c.dom.Document doc = builder.parse(is);
    is.close();
    return doc;
}

GUI vs Command Line

For those in software development and aiming at more, my following observation:

when one launches a task in GUI and it gets by some means frozen, there is no way to painlessly stop that particular task but kill the entire GUI.

when one launches a task in the Command Line and you suspect it got frozen, you simple stop the task by CTRL+X or CTRL+Z or CTRL+C (any key combination that does it).

The point is: I would aim at multithreading (or even creating separate processes) of consuming tasks so that I could easily undo any task-related actions. I would be really happy to have a functionality where I can press CTRL+Z to undo any action be it new thread/process for a task or typing a letter in a sentence.

Unusual Billie Jean video

It has been a fun and honour to be a fan of Michael in my youth. This video looks very unusual and at the same time unique vs what you can find for the video search query "Billie Jean":

More Videos

credits: http://www.michaeljackson.com

The most lengthy PhD dissertation,

I have ever seen is here.

Verbmobil

An exciting promotional video of Verbmobil - machine translation system for "spontaneous speech in face-to-face situations".

Sometimes looking at history makes you excited.