https://brlcad.org/w/index.php?title=User:Popescu.andrei1991/devlogs2012&feed=atom&action=historyUser:Popescu.andrei1991/devlogs2012 - Revision history2024-03-29T10:08:49ZRevision history for this page on the wikiMediaWiki 1.32.2https://brlcad.org/w/index.php?title=User:Popescu.andrei1991/devlogs2012&diff=7200&oldid=prevPopescu.andrei1991: Added 2012 devlogs2014-06-04T08:30:42Z<p>Added 2012 devlogs</p>
<p><b>New page</b></p><div>==Personal Info==<br />
Hello, my name is Andrei - Constantin Popescu, I usually go by Andrei. I am a second year undergraduate at Polytechnic University of Bucharest,studying at the<br />
computer science department.<br />
<br />
==Contact== <br />
I can usually be found on IRC on #brlcad - freenode, I have various nicknames, depending on availablity : andrei, andrei_, andrei__.<br />
The quickest way to reach me on e-mail is the address:<br />
popescu.andrei1991@gmail.com - I read emails on this several times a day.<br />
My github account is : https://github.com/pandrei <br />
<br />
I will use this page to keep track of my progress on a daily basis.<br />
<br />
==Project Summary==<br />
Firstly, here can be found my official proposal.<br />
https://google-melange.appspot.com/gsoc/proposal/review/google/gsoc2012/popescuandrei/4002<br />
<br />
The purpose of a package managing library ( or tool) is to pass around packages <br />
containing data from brlcad tools. The packages are sent via TCP and as a result<br />
are using TCP header.<br />
TCP -- http://en.wikipedia.org/wiki/Transmission_Control_Protocol<br />
<br />
My current knowledge is that the TCP Payload[1] has a size of 64kb, at least<br />
in the linux kernel. There has been a reported incident where libpkg fails to send 24k+ packages.<br />
[1] Payload - TCP field containinig effective data (user data, for example).<br />
<br />
I believe this is the reported incident : http://permalink.gmane.org/gmane.comp.cad.brlcad.devel/959.<br />
<br />
Tpkg will be used for testing the package issue. <br />
I am currently researching dstat and ifstat, found via <br />
apt-cache search ifstat<br />
dstat - versatile resource statistics tool<br />
ifstat - InterFace STATistics Monitoring<br />
<br />
The curent state of brlcad libpkg is a basic client-server protocol.<br />
<br />
<br />
<br />
==GsoC 2012 progress==<br />
<br />
So far I have managed to compile and install brl-cad on a 32-bit Archlinux.<br />
I am focusing on the tpkg command to see what exactly happens and properly submitting the global removal patch.<br />
<br />
<br />
Tests have been run 3D version on a remote machine. The graphics are present on here on the log. However, after more discussion it was decided there is too much noise in 3d mesh.<br />
<br />
Currently, tests have been performed for package size from 2048 to 4194304 ( 2048 ^ 2 ). Tests have been performed on a 8M file. <br />
This is the graphic obtained by testing : http://i.imgur.com/H5Q2u.jpg <br />
<br />
TODO : add some github link with files and numerical values.<br />
<br />
There is a second test, running package size from 1 to 2048 on a 8M test.<br />
It has not finished at the moment of writing. However, a graph was drawn with some partial data ( 1 - 95 package size ). It's correctness needs to be checked but it does seem to look like an exponential curve.<br />
<br />
TODO : add details and graphics when the script finishes.<br />
<br />
I m currently working on fixing the unit test. <br />
<br />
== Performance measuring ==<br />
<br />
The 3d graphics have finished.<br />
<br />
File size increase as powers of two series from 1 to 8MB<br />
<br />
Package size 1 - 2048<br />
<br />
http://i.imgur.com/OPum5.png<br />
<br />
Package size 2048 - 4194304<br />
<br />
http://i.imgur.com/fH4rL.png<br />
<br />
Package size 1 - 4194304<br />
<br />
http://i.imgur.com/AXUpN.png<br />
<br />
After converting data into CSV format and using GNU plot I have obtained the following graphics : <br />
<br />
1 - 2048 : http://i.imgur.com/tnWAa.png<br />
<br />
2048 - 4194304 : http://i.imgur.com/isNXL.png<br />
<br />
Currently I m working to see if I can combine the two files. From what I read so far this seems rather complicated.<br />
<br />
== Daily development log ==<br />
<br />
<br />
== Weeks 1 - 3 ==<br />
21th May - 8th of June - exams period, minor work on tpkg patch and on the<br />
red - black tree test unit.<br />
<br />
9th June - working on obtaining commit access asap. Currently developing the<br />
performance shell script.<br />
<br />
10-11th June - not much work , documenting regarding bash programming.<br />
<br />
== Week 4 ==<br />
<br />
12-13th June -finishing the shell script, applying feedback given by Erik on the tpkg parameter patch.<br />
<br />
14th June - submitting the final version of the shell script.<br />
<br />
15-17th of June - learning K&R indentation, learning about POSIX. Finishing the tpkg script and the tpkg.c modifications.<br />
<br />
18th of June - unsuccesful attempts to sync the server and client ( To start the client only after server started) using signal handling, docummenting about signal handling<br />
<br />
19th of June - unsuccesful attempts to achieve the above by using inter-process communications, understanding thread concepts, docummenting about the basics of fork() , pthreads_<br />
<br />
== Week 5 ==<br />
<br />
20th of June - sync the server with the client by using a loop to detect a key string (while "Server_ready" isn't found in the log_file the client does not start. Having issues with redirecting output from a background process<br />
<br />
21st of June - finished tpkg script for localhost ( both client and server are on the same machine), developing the remote one. (Using two different machines)<br />
<br />
22nd of June - getting some hints from #bash, reading about ssh, working on the ssh remote script(password-less ssh login process). Having issues with the ssh session not exiting properly.<br />
<br />
23rd of June - script finished, full functional started script on two machines<br />
setting up log files / reading a bit about sampling necessary for graphic( how many points are enough )<br />
<br />
24th - 25th of June - scripts running time, realized I could have done them a lot better and faster. No notable work done in these days. ( a script using small files ran twice and one using large files( up to 700 mb ) ran once.<br />
<br />
26th of June the total dimension for the log files was 8.4Gb on server machine as well as on client ( minor difference) purging the files from irrelevant data.<br />
read about sed and awk.<br />
Compared the first two tests( which use identical parameters) and found no notable difference. Starting to read about and write the octave script. Reading about interpolation and how can I use it if I don t know what points am I missing.<br />
<br />
27th of June - asked for some mathematical hints on #octave - freenode.<br />
Finished graphs currently without interpolation. Looking up bu_bomb.log but they are empty. Re-run the script for a small number of iterations still found no error. <br />
<br />
== Week 6 == <br />
<br />
28th of June - uploaded data. upated logs.<br />
<br />
29th of June - Reading about google test, planning on what testing framework to use for libpkg client - server.<br />
<br />
30th of June - still stuck with the OOP from google test.<br />
<br />
1st of July - learning / understanding how to use google test for this was taking too long, finding another way.<br />
<br />
2nd of July - initially created the unit test as a shell, after further digging, decided to change the plan and make it a .c unit test.<br />
<br />
3rd of July - finished unit testing. uploaded unit test on sourceforge. <br />
<br />
4th of July - minor work, mostly look up things I already did and see where and what I can improve.<br />
<br />
== Week 7 ==<br />
<br />
5th of July - Issue with wallclock, script was modified to counter the wallclock instability. Single variable tests were run.<br />
<br />
6th of July - issues with the script, for some reason ssh was not working. Adjusting that. Finished all the issues with the scripts / tpkg. Started 2d graphics ( 1 - 2048 ) and ( 2048 -> 2048^2).<br />
<br />
7th of July - 2048 -> 2048^2 testing finished, graph drawn. Minor analysis on it. Drawn a graph with the partial data from 1 - 2048 one, does seem incorrect. Will be reworked.<br />
<br />
8th of July - Restarted 1 - 2048 test. Discussing further tests and how I should proceed. Minor reading about unit tests in networking<br />
<br />
9th of July - Scripts still running, wrote unit test. Received feedback started applying it. Monitoring the values obtained from the tests. The " 3D graphic " test failed ( script execution ended ). It was an script error not a tpkg error.<br />
Fixed it and resumed tests.<br />
<br />
10th of July - working on the test unit, running tests. <br />
<br />
11th of July - Graphed the results of one test<br />
<br />
== Week 8 ==<br />
<br />
12th of July - discovered an error in testing. Previously calculated average was taken into consideration when it should have not, restarted scripts. Got two more machines ( installing archlinux + brlcad on them )<br />
<br />
13th of July - fixing HACKING issues on previous patches. Providing 1-2048 Graph.<br />
<br />
14th - 15th of July - minor work, waiting for tests mainly.<br />
<br />
16th of July - finished tests, graphed results. TODO:// Complete with rest<br />
<br />
17th of July - obtain data in CSV format, attempt to check scripts correctness, minor reasearch about gnuplot<br />
<br />
18th of July - plotting data in a single file, taking a look into pkg.c and pkg.h to decide what unit tests to write<br />
<br />
== Week 9 ==<br />
<br />
19th of July - didn't manage to work anything at all, had somehardware(hdd)failure, reinstalled OS.<br />
<br />
20th of July - started working on test_pkg_permserver, some documentation regarding unit tests. <br />
<br />
21th of July - finished test_pkg_permserver, minor HACKING reedit, started working on test_pkg_getclient.<br />
<br />
22th of July - trying to solve issues with blocking calls. Document about potential solutions such as threading / signal handling or select ()<br />
<br />
23th of July - abandoned test_pkg_getclient for the moment, moving to <br />
test_pkg_open, had some issues. Pkg_open was not exiting correctly.<br />
<br />
24th of July - finished test_pkg_open. Moved to test_pkg_stream, almost finished but there are some issues with the structure of the test. ( checking only pkg_conn header)<br />
<br />
25th of July - Clean-up on the current test units, searching for other potential candidates. Taking a look into pkg_send and pkg_send2.<br />
<br />
<br />
== Week 10 == <br />
<br />
26th of July - asked for guidance on IRC, uploaded on sourceforge.<br />
<br />
27th of July - None<br />
<br />
28th of July - still searching for methods to escape a blocking call,<br />
none found using stdout as file descriptor. finished writing test_pkg_getclient<br />
<br />
29th of July - fixing indent issues and such on test_pkg_getclient, started working on test_pkg_stream. finished coding test_pkg_stream.<br />
<br />
30th of July - fixing minor issues with test_pkg_stream. started working on test_pkg_flush. Finished working on test_pkg_flush( tho there are some issues)<br />
<br />
31st of July - started working on pkg_block and pkg_suckin, investigating some issues regarding pkg_conn<br />
<br />
1st of August - finished pkg_block / pkg_suckin, checking out contributor quickies in order to obtain commit access<br />
<br />
== Week 11 ==<br />
<br />
2nd of August - done test_pkg_close and started working on test_pkg_waitfor<br />
<br />
3rd of August - finished working test_pkg_waitfor. reading /understanding pkg_send.<br />
<br />
4th of August - worked on test_pkg_send. Finished it. Minor work on pkg_2send.<br />
<br />
5th of August - finished working on pkg_2send.<br />
<br />
6th of August - taking a look into pkg_bwaitfor, no other work done.<br />
<br />
7th of August - started working on pkg_bwaitfor, finished it. however still some minor issues with it<br />
<br />
== Week 12 ==<br />
<br />
8th of August - None<br />
<br />
9th of August - Minor programming issues. None other<br />
<br />
10th of August - read pkg.c code, attempted to fix fail cases discovered in<br />
unit tests.<br />
<br />
11th of August - continued to work on fixing fail cases ( difficulty in finding out which ones are legit and which ones I should fix ) PKG_CK for example, aborts execution.<br />
<br />
12th of August - cleaning up unit tests, making them ready for upload<br />
<br />
13th of August - Converting tests into patch format and uploading them , fixed some more fail cases, adjusting code.<br />
<br />
14th of August - issues with a segmentation fault in pkg.c code, working on fixing it.<br />
<br />
== Week 13 ==</div>Popescu.andrei1991