Tuesday, July 31, 2007
First bits of PyQt
The program is less than 170 lines of code. You can try it if you are registered at vkontakte.ru, but the program doesn't matter now.
I am very pleased with simplicity of creation of portable rich GUI applications with PyQt, but as always there are some issues on Windows. I created the program on my favorite old Linux box with vim, so there I had no problems with PyQt4 installation and no need of packaging, because the program consists of one file and three icons.
Then I decided to try it on Windows. First of all I installed newest PyQt 4.3.0. This new version is now packed in self-installing archive, which includes Qt and Eric4 IDE, so now you shouldn't have problems which I described in this post.
After that the program worked perfectly, but I needed to create standalone version for people who don't have Python and PyQt on their PCs.
This is rather simple task, but I've spent several hours fighting some issues.
I used py2exe to create standalone executable. The problem was to include PyQt into it. The idea is simple: I needed to specify that PyQt should be included. So I used solution from py2exe.org. I created file called setup.py with following contents:
from distutils.core import setup
import py2exe
setup(windows=[{"script":"tray.py"}], options={"py2exe":{"includes":["sip"]}})
Then I ran python setup.py install py2exe.
But I received errors like "module import failed: _qt". So I decided to mention inclusion explicitly: python setup.py install py2exe --includes=PyQt4,sip. It didn't help, but error message changed to "cannot find QApplication". Then I tried to reorganize imports in my code:
import sys
from PyQt4.Qt import *
from PyQt4 import QtCore, QtGui
gave correct result, and the executable was created.
I believe that this problem with imports is temporary and will be fixed in near future.
So then I used Inno Setup to create setup package and it was really easy.
Conclusion
It is really easy to create portable applications with great GUI with PyQt4.
The only trade off is the size of installed application on Windows: for my 170 lines of code project I've got 5 Mb setup package, which extracts into 17 Mb application.
The reason is simple - lack of common packaging system with dependencies on Windows, so the program needs it's own copy of Python and PyQt.
Friday, July 13, 2007
My PCs
For those, who haven't left yet.
My father is a software engineer, so there is nothing unusual that we'd got first home computer as soon as was possible. It was in 1990 or 1991, when the Soviet Union still existed, so there were some problems with foreign devices. Our first computer was hand-made ZX Spectrum clone, which was assembled from different foreign and Russian chips by my father's friend. It had rather peculiar hand-made keyboard, and used Kometa-212 tape-recorder for data storage:
and monochrome TV Kaskad-203 as the display, which looked like this:
I'd just started to play chess in local center for youth creativity and enjoyed playing with this computer very much. I was unable to beat it at that time, since I was only six or seven years old :)
Also I tried to do some programming in built-in Basic, but without much success, since I was unable to save entered program on the tape.
I will try to find and photograph remains of that computer.
It's a pity that couple of years after our old TV ceased to function, so we bought another TV, which was incompatible with computer output.
So for couple of years we had no working computer at home. I should mention, that the beginning of 1990s was rather hard time in Russia, so IBM PCs were to expensive for us. They cost more than $1000, and it was very big money. The average wage at that time was about 50-100 USD a month.
When I was eleven, I started study programming in Palace of Youth Creation. I was taught Turbo Pascal 7.0. So I was really happy when we bought a computer as a New Year gift at the end of 1996.
It was rather good and expensive machine with AMD K5 75 MHz processor, 8 megabytes of RAM, 600 megabytes hard drive and brand new Windows 95 OS and 14" CRT monitor GoldStar 1468.
I still use this monitor, which has rather decent picture quality and 800x600/75Hz resolution.
We also bought a printer HP DeskJet 400, which is really cool, I used it last time just two years ago, when I had no time to buy a cartridge for new printer.
I enjoyed playing Warcraft II and programming in Turbo Pascal. But the OS was a complete disaster, it showed Blue Screen of Death several times a day. May be it was due to the lack of system resources, since OS became more stable when we added 8 more megabytes of RAM a year later, or may be because the system was cracked, not bought legally. As you remember, couple hundreds of bucks were a lot of money in Russia in that time :)
In the end of 1997 we added 8 megs of RAM, sound card, CD-ROM and 1.2 Gb hard drive to our PC. Nevertheless, by the end of 1998 it became completely obsolete.
So we "upgraded" it by replacement of main board, processor, RAM, and hard drive. We used the box of old PC, and also bought new LG Flatron 795FT monitor.
New computer was pretty cool, it was based on AMD K6-2/350Mhz processor, with 64 megs of RAM (128 more megabytes was added a year later), 6 Gb Seagate Medalist hard drive and great USR Courier 56k modem. Modem was really good on our old phone line. We had almost no breaks, but rather slow (33,6k) connection because of old hardware at local exchange.
In 1999 computer section in Palace of Youth Creation switched from DOS/Windows 3.11 to FreeBSD, so I started to study C, C++ and Perl, and installed RedHat Linux 5.2 and later 6.0 and 6.2 on our first hard drive, which was 600 megabytes. It's funny that there were enough space for X11, KDE, XEmacs, LaTeX, kernel sources and so on. I also enjoyed Fallout 2 game.
In the end of the year 2000 we decided that we need new PC, so we bought completely new AMD Athlon XP 1733+ MHz with 256 megs of RAM and 40 Gigs IBM hard drive. We plugged newer LG monitor into new PC, and old GoldStar to older one. So for the first time we had more than one working computer :)
Well, later we added new hard drives to new PC, 512 megabytes of RAM, CD-RW/DVD-RW drives, new Canon PIXMA iP4000 printer, Epson scanner and so on, but that is not very interesting. In the beginning of this year 40 Gb IBM hard drive on older PC failed, so that computer became Linux only box. I installed KUbuntu 7.04 on it and I use it now everyday for software development, and it is rather fast :) It can host Apache, MySQL and PHP, also I do C++ and Python development on it. It even can play Youtube movies :) The only thing that I've done for optimization was the replacement of KDE with Fluxbox.
Thursday, July 5, 2007
QFT on non-commutative spaces
Fiore and Wess in their article ”On full twisted Poincare’ symmetry andQFT on Moyal-Weyl spaces” shows that non-commutative QFT withproperly enforced ”twisted symmetry” is equivalent to ordinary commutativeQFT. They start from enforcing space non-commutativity in most simleway:
| | (1) |
Where is a constant matrix. So the limit
gives ordinarycommutative space. Then algebra
generated by
is equivalent to algebra offunctions
on commutative space with deformed product, called star-product.Such star product is introduced using twist
:
| | (2) |
Properly chosen gives
| | (3) |
as required.
There are whole classes of equivalent twists, which lead to the same starproduct, so the theory doesn’t depend on the choice of particular twist. ThenFiore and Wess show how to construct QFT starting with . To obtain properself-consistent theory they have to change Poincare-invariance with invariancewith respect to deformed Poincare group, obtained from deformation of Poincarealgebra with the same twist
, as was used to deform algebra of functions
.
After the introduction of Wightman axioms and study of Wightman andGreen’s functions it turns out, that this functions coincide with their undeformedcounterparts at least perturbatively. So deformed Poincare symmetry works as”compensation” of space non-commutativity, and theory gives no newphysics.
But there are another possibilities. We can use more complex non-commutativespace, for example, we can start from deformation of Poincare symmetry to-Poincare, then introduce a twist on
-Poincare and build non-commutativespace with twisted
-Poincare symmetry.
I hope to study this case in details in the next several days.
Thursday, June 28, 2007
Creating simple yet powerful anonymous downloader in Python
Python is a very convenient language for this task because it has Interactive Mode (or REPL - Read-Eval-Print-Loop). So we can start Python and write something like this:
>>>import urllib,re # some modules which we will need later
If target web site has some kind of index page, we should start from getting it and parsing it into items:
>>>index_page=urllib.urlopen('http://example.com/index').read() # we have read contents of the page
>>>items_to_get=re.findall('some regexp',index_page) # now we have list of all items to download, regexp may be something like '<a href=\'([^']*)\'>'
>>>out=open('index','w') # let's save all the items for future use
>>>for item in items_to_get:
... out.write(item+'\n')
>>>out.close()
It is really simple!
Now we can get all the pages. Let's suppose, that items are simple pages, something like 001.html
>>>for item in items_to_get:
... open(item,'w').write(urllib.urlopen('http://example.com/%s'%item).read())
...
This two lines of code will download and save all the pages into files like 001.html
But there are two big problems. First, we download only one page a time, so it will take a lot of time to get all the pages even in case of broad connection because of delays on sending requests, writing pages to disk and so on. Second and more important: site admin can notice that someone (you) tries to download the whole site and ban your IP.
So we need to write real piece of code.
I will write it in separate file getter.py:
1 #Let's import something
2 import os,urllib,urllib2,socket,sys,re,threading,time,random
3
4 max_thread_num=10 # we will download in ten threads!
5
6
7 # Let's create class for a thread of download
8 class ProxyThread(threading.Thread):
9 def set_params(self,proxy): # proxy address
10 self.proxy=proxy
11
12 def run(self): # main method of thread
13 # first we create proxy_handler to handle connection with anonymous http proxy
14 proxy_handler = urllib2.ProxyHandler(self.proxy)
15 auth = urllib2.HTTPBasicAuthHandler()
16
17 # then we build opener, wich works through the proxy
18 opener = urllib2.build_opener(proxy_handler, auth, urllib2.HTTPHandler)
19 #some fake User-Agent (may be useful for some sites)
20 opener.addheaders = [('User-agent', 'Mozilla/5.0')]
21 #Let's now check connection with proxy by getting some data from google
22 try:
23 urllib2.urlopen('http://google.com').read(5)
24 except:
25 # this proxy doesn't work, we will try another.
26 #We need to increment number of available threads.
27 #Simply use global variable
28 global max_thread_num
29 max_thread_num+=1
30 return
31
32 # this proxy is good, so we get items, wich are needed to be downloaded
33 items_to_get=open('index').readlines()
34 # we shuffle list so that different threads try to download different items
35 random.shuffle(items_to_get)
36
37 for itemstr in items_to_get:
38 item=itemstr.rstrip() #strip newline symbol
39 if os.path.exists(item): #Already downloaded ?
40 continue
41
42 #Let's try to download something :)
43 try:
44 data=opener.open('http://example.com/%s'%item).read()
45 #CoDeeN proxies give greetings page on first request, so we need to skip it
46 if not re.search('CoDeeN',data):
47 open(item,'w').write(data)
48 except:
49 #If there was some error during download,
50 #we will remove incompletely downloaded item,
51 #and skip this proxy, because majority of errors are proxy errors:
52 try:
53 os.remove(item)
54 except:
55 pass
56
57 global max_thread_num
58 max_thread_num+=1
59 return
60
61 global max_thread_num
62 max_thread_num+=1
63 return
64
65 # Well, it was quite a long method mostly because of error handling.
66
67 #Now small piece of code wich gets list of proxies and starts threads:
68 socket.setdefaulttimeout(120) # increase connection timeout because some proxies respond after rather long time
69 #We will go trough all 50 pages of free proxy list at http://www.samair.ru
70 for num in range(50):
71 # We get page of proxies
72 proxy_page=urllib.urlopen('http://www.samair.ru/proxy/time-%02d.htm'%num).read()
73 # and search for all proxy addresses on it
74 proxies=re.findall('<span class="proxy\d*">(\d+)</span>.<span class="proxy\d*">(?P<n2>\d+)</span>.(\d+).(\d+):\s*(\d+)',proxy_page)
75 # rather complex regexp :)
76 for proxy in proxies: # try to start thread for each proxy
77 while max_thread_num<=0: # all 10 thread have started, so we need to wait
78 time.sleep(1)
79 # Then we can start a thread:
80 prox={'http':'%s.%s.%s.%s:%s'%proxy}
81 pt=ProxyThread()
82 pt.set_params(prox)
83 max_thread_num-=1
84 pt.start()
Then we can start the program python getter.py.
Well, it works!
Almost :)
I've written no ending condition, so this code will end not when all the items will be downloaded, but when all the proxies will be checked. Anyway you need to decide which items you should download and what to do in case of errors.
Well, it looks like the next post can be about spam bots or other kinds of malware :)
Nonetheless, I believe that you won't use the code for illegal actions :)
Friday, June 22, 2007
Fit2PDA moved to Google Code, new project started
All the source code is in svn now. Also I've written some help pages for project and uploaded build for Windows.
Linux build will be ready tomorrow.
So please feel free to download try and write bug reports
I will be happy to get new ideas and patches too!
New project vkontakte.net.ru
Last two weeks there was very little activity on Fit2PDA from my side, because of my new web project vkontakte.net.ru.
It is devoted to research on most successful Russian social network vkontakte.ru. I created small program in Python to draw maps of connections between people in this network. Web site has wiki for discussion of new ideas.
I need to warn you that vkontakte.net.ru is and will be in Russian only. Anyway why one not reading Russian can be interested in research of Russian social network :)
You can visit vkontakte.net.ru for more information if you read Russian.
Monday, June 4, 2007
Eric IDE on Unfriendly System
So I decided to install it on my work PC.
Unfortunately it's running Windows XP, and I can't change it. And those OS doesn't have normal package manager.
To install Eric I needed Qt, Python, PyQt, QScintilla.
I had opensource Qt (installed with MinGW) and Active Python installed on my PC.
So I downloaded PyQt, QScintilla and Eric.
I installed binary package of PyQt and started to build QScintilla with MinGW. The build was successful, but python bindings didn't installed because of some error with SIP. So I tried to build SIP separately. It wasn't build because of some unknown linking errors with Python libraries.
I searched trough the internet, but was unable to find solution.
So I decided to reinstall everything.
First of all, I downloaded Python from python.org and installed it (it was what really helped).
Then I reinstalled Qt.
Then I downloaded source package of PyQt, unpacked it and run
python configure.py
make
make install
The build completed with no errors.
Next I downloaded QScintilla source and unpacked it. Then I went to Qt4 subdirectory, and run
qmake qscintilla.pro
make
make install
After that I went to Python subfolder of QScintilla source, and ran
python configure.py
make
make install
When the build was completed I had all needed to run Eric4.
(Also SIP is included into PyQt, so one don't need to build it separately).
Then I unpacked Eric and run
python install.py
eric4.bat
It started after all!
Eric is really great, but I've met the problem on second run - it haven't started because of some error in debugger configuration. So I had to turn remote debugging and passive debugging on through eric-configure.bat. After it the IDE started normally.
Saturday, May 26, 2007
Time to upgrade Ubuntu on my home boxes
I have two PCs at home: old AMD K6-2/350 MHz with 40 Gb dying primary hard disk (Windows 2000 is installed on it :)) and 6 Gb secondary drive, where Ubuntu 5.10 was installed, and newer Athlon XP 1800 with Ubuntu 6.10 on it.
It's a pity that you can not use live/install Ubuntu cd for upgrades :(
So I've decided to do dist-upgrade on newer PC and completely reinstall system on older. I've chosen Kubuntu for old box, because it includes Konqueror, which is more lightweight than Firefox.
Several hours ago I've back-upped all important data from my older pc and started installation of Kubuntu. I've already met several problems:
- I need to write acpi=force in the boot options of live-cd, because BIOS was released before the year 2000.
- Installer was unable to format partition into XFS because of some unknown error (possibly connected with dying primary drive), so I had to create partitions by hand using QParted.
- May be I'd better to write hda=noprobe boot option to prevent errors from accessing dying primary drive, but I haven't done it
Now the installation process is 94% complete.
And almost half of required packages have been downloaded on my newer PC through my slow 200 kbit ADSL connection.
So keep on touch with me, I will report the progress and problems!
Update 1:
The installation of Kubuntu is finished, now it is time to test the system and install additional packages.
I plan to install:
- LaTeX
- maxima
- developer's tools
- java
- Mp3 and dvd playback
- Open SSH
Update 2:
Installation of mp3 playback was really easy - I tried to play mp3 file and Amarok asked me to install mp3 support, I answered yes and in couple of minutes it started to play.
There were no problems with OpenSSH server too: sudo apt-get install openssh-server
But Adept apt front-end is really slow :(
Well, I installed almost everything I wanted, only couple of packages left.
And download of packages for upgrade on newer PC has not finished yet.
Update 3(next day):
All updates and installations are finished, and now I'm happy with two up-to-date systems.
Rhythmbox on my newer PC now plays last.fm. I'm also happy with kopete LaTeX plugin
Wednesday, May 23, 2007
On educational system
The author of this article works in area of Computer Science, but the tendency seems to be universal both in UK and US. (You can look, for example, through this post)
So why we in Russia should be interested in problems of far richer countries?
Our educational system is in decline because of enormous and long-lasting lack of money. But modern reformation efforts introduce new dangers.
Today we are accepting participation in so called "Bologna process", which will inject to our education some problems of British system, mentioned in the first article. The most obvious one is modularity, which allows students to choose courses to learn. It leads to lack of interest in harder core courses, and I can feel it myself. Today I have passed in two of four courses which were given in our department (of QFT), and I have no need to pass the tests on other two, which are far harder.
Nevertheless, "Bologna process" is a minor threat comparing with government educational policy.
By the way, our problems will inflict western education too, since it is supported by immigration from our country.
The only hope is academic emigration from China and other developing countries.
Friday, May 11, 2007
Draft of the article
This is only first draft, so I will be happy to get your opinions and corrections, especially on my poor English.
This articles is intended to an advertisement of my project Fit2PDA :)
Wednesday, May 9, 2007
Fit2PDA
So it is now called Fit2PDA.
I think that the name is apt to the program :)
I've done some additional improvements in GUI, so now you can see (rather accurate:) progress bars during conversion and you can convert not only PDF/PostScript, but also Djvu.
Big number of free scanned ebooks can be found in Kolhoz library and eDonkey2000 p2p-network.
By the way, there are 2 good compatible clients for eDonkey: eMule for Windows and aMule for Linux. So, if you use both operation systems, you can continue your emule downloads in amule. You just need to select the same working directory!
The book "Innovator's Dilemma"
I read several times that it is "the best business book ever written" an so on.
Well, I don't know is it true, I haven't read a lot of business books.
The book is really interesting, but the main idea of the book, that there are so called "disruptive technologies", which can't be handled by existing businesses seems rather natural.
Why else new businesses would appear after all?
I was able to download this book for free from Internet shop bizbook.ru, because they give 20 pages preview of the book for free (in very poor quality images).
So I've used number of anonymous proxies and downloaded the whole book.
I had to do it because I was unable to find the book in usual warez sources and p2p networks.
It a pity that I had to do such a thing myself, because it means that we don't have access to all the books worth to read even through pirate's networks.
By the way, I used my program Fit2PDA to convert this book, and it worked perfectly :)
Wednesday, May 2, 2007
First "public" version of "qtsplitter"
First completely working build for Windows is here
Simply download and extract archive, then run converter3.exe , choose pdf-files to convert, select correct screen width of your PDA and convenient resolution and press on button with red point. Then take a cup of coffee and wait while your papers or books are converted.
You will get iSilo-friendly html files, which could be easily converted to Palm-powered with iSiloX, or viewed with Pocket IE on PocketPC.
So I will be happy with any feedback from you!
Also I will be happy to get help with UI design and Linux package creation.
And the final question: do you think that "FitToPDA" is a good name?
Tuesday, April 17, 2007
New version of "qtsplitter"
I discovered quite simple solution - we don't need to recognize text, we can just reformat the images of pages to feet to PDA screen.
So that's all about history and motivation. Now let's talk about new version. In this version I incorporated deskew algorithm which automatically rotates image to make text horizontal. Also I reworked splitting code, and now the program is able to split colorful images (though, it uses only black-and-white data to find lines of text) and it tries to preserve under- and over- lines.
I was thinking a lot about coding style when writing this code, because I didn't have sufficient C/C++ practice last couple of years. So I will be happy to hear your suggestions on that matter.
Also I want to give sounding name to this program, so I declare some kind of contest for the best name, and I'm waiting for your proposals.
And one announcement: next version will have a GUI, so I plan to make it public. I will be happy with any kind of support.
Update:
I forgot to mention sexy example
Thursday, April 12, 2007
Just finished reading the book
It is old but excellent book on programming in GNU/Linux environment. It starts from brief description of everyday developer's tools such as emacs, gcc, gdb, make and then leads to core topics of Linux development. It covers main system calls, IPC, networking, usage of shared libraries and plugins, error handling and so on.
So it is really useful for experienced developers and even for beginners because the text is very clear. There is no information on GUI development and minor libraries, so the book is very useful even six years after the publication.
It can be downloaded from the official site
I was doing some marks when reading, so I created some kind of reference card page on some covered in the book. You can find it here
Tuesday, April 10, 2007
About my projects
I use it not only as photo album, but I also put here information related to my current free-time projects.
Now I try to make my qtsplitter program usable. Now it can be used as command line tool under both Windows and Linux.
This program splits pages of scanned books into lines and split lines into parts so that they fit into PDA display. You can see how it works here .
You can try the program yourself if you are really interested and know how to build Qt projects.
I would really appreciate if somebody could make and upload statically linked version for Windows.
Also I started to collect useful tips on Vim editor here. After all my site is really apt to such kind of projects because it is wiki-based.