User:Shainasabarwal/GSoC15/logs

Community Bonding Period

• Wrote code for new scadlexer in scadlexer.h and scadlexer.cpp files.
• Fetch the colors in scintillaeditor.cpp file for new scadlexer and it is successfully working
• Wrote functions on highlighting keywords and highlighting comments.
• Tested with Qt4 as well as Qt5
• Wrote function for highlight multiline comments
• Wrote function to highlight numbers
• Trying to search any better way for lexing approach using QSciLexerCustom class.

Coding Period

25 May

• Discussed and researched about different tools that can be used in scad lexer.

26 May

• Chosen lexertl to be used for scad lexer.
• Started reading lexertl documentation

27 May

• Lexertl in boost is not the latest version, so as per its latest documentation it doesn't work, trying to read code of lexertl included in boost

28 May

• Contacted Ben Hanson, the person who wrote lexertl and its documentation and he suggested to use the latest version of lexertl and not that included in the boost, because the latest version has some bug fixes and it also supports unicode. Also it is header file library so, we won't be needing to make any change in installation script
• Discussed about this with my mentors
• Added lexertl header files in openscad new branch lexertl

29 May

• Read lexertl documentation and examples at https://github.com/BenHanson/lexertl/tree/master/examples and tried to add something into scadlexer files to test its working.

30 May

• Added rules for numbers, operators etc and divide code into tokens.
• Wrote the code for lookup table.

1 June

• Successfully started highlighting keywords using lexertl tool in scadlexer.

2 June

• Added rules for single line and multi line comments.
• Multi line comments are not working as expected.

3 June

• Read more about regular expression to have an efficient rule for single line and multi line comments.

4 June

• Editor became slow. Trying to figure out what's the problem with it. May be the styleText function is being called again and again and so with it all rules of lexer are being called again and again on making any change in the editor by the user.

5 June

• Discussed with mentors and defined all the rules in constructor instead of styleText function and this way, editor got its normal speed.

6 June

• Discussed with mentors about problems in comments and keyword highlighting and started writing tests.

8 June

• Wrote a C++ program using lexertl library with simple rules as testing.

9 June

• Get the tests written checked by mentors and made more improvements.

10 June

• Preparation for viva-voce in college, generating report and presentation

11 June

• All day spent in college for waiting and delivering presentation to teachers and students

12 June

• Read more in lexertl documentation to write comments as another state machine.

15 June

• Discussed with mentors about writing tests, ie, reading scad script from an input file, mark the keywords, numbers etc and then put the output in another file.

16 June

• Wrote test in C++ using lexertl library, defining various rules, which puts in the output in output file and get it checked by the mentors.

17 June

• Compiled test suite of openscad and read its documentation.
• Wrote lexertest in openscad test suite.

18 June

• Made a new class Lex that can be used commonly by the tests as well as the scadlexer.
• Added lexer rules in the class but it is not resulting any kind of highlighting.

19 June

• The new syntax of stl vector class used to push all elements in one line could only be compiled using C++11 reference, so it didn't work while making openscad tests. The default syntax of pushing elements will increase a lot of line of code as a lot many elements need to be inserted.

• To solve above problem simple C style array are used to store all elements.

• Keyword lists in scadlexer were removed as now it will be using the arrays defined in lex class.

• Still not resulting highlighting.

20 June

• lex file object is added in lexertest of openscad.
• lex.cc file reference is added in CMakeList and solved reference error.
• Removed the vectors defined in tests, as it will also be using the arrays defined in lex class.

22 June

• Number of elements in each array is calculated by sizeof function and put in defineRules function of lex class as an argument.
• Added more rules in lex class
• Solved more errors and highlighting of different tokens start working using lex class.

23 June

• Put the while loop which is reponsible for highlighting tokens according to the defined rules into the Lex class so that it can also be used commonly by lexertest test and scadlexer class.

24 June

• Made LexInterface as abstract class to call the highligting() function of different classes(scadlexer and lexer in lexertest file) in Lex class.
• Removed errors in lexertest file and compiled the tests
• Discussed with mentors about writing the test cases

25 June

• Wrote 2 test cases and write the file path in CMakeFile.txt file to pass them to add_cmdline() function so that respective python script that take those files and input, run the lexertest and generate respective output.
• Pass both tests

27 June

• Multiline comments were causing problem in main lexing, as the value of start becomes the start of the line when some changes are made in particular line.
• Trying to figure out more such problem in lexing.

28 June

• To test this multiline commenting thing in lexertest, the value of the start is made changed but currently, it is not affecting the output of the test.

Trying to figure out the problem.

29 June

• In highlighting function, the start was supposed to be used to change the pointer from where the file start to be changed. But recently, the variable start was not used in the function at all, so it was not causing any change.
• Trying to figure out, how will it be used in the function

30 June

• Trying to use the seekp function to put the start pointer at different location, but as the highlighting function is being called again and again at every token whereas the start is remained fixed. It is not giving the expected result.
• Trying with more functions to generate the expected output.

1 July

• Use tellp to check the current location of pointer, it is more than start, then highlight it otherwise, right the script as it is.
• But it didn't behave same as openscad qscintilla editor. Discussed with mentors about the problem that, whole source string need to be divided into substring starting from start.

2 July

• Divide the source string into substring started from start, but still the index of token in lexertest is different than in openscad editor because of tabs as openscad editor take tab as number of space tokens whereas lexertest takes tab as single token.
• Change tab to spaces in lexerinput file, now editor and tests are behaving same, having same token index.

3 July

• Scad Lexer is facing the problem of multiline comments, as the start variable in scadlexer is the start of current line, instead of start of document. So some changes are made within multiline comments, it gets highlighted again according to other rules instead of being the part of comment.
• The behavior of the keywords are also not so good. Discussed with mentors about these problems and tried to find out the solution.

4 July

• The problem of unusual behavior of highlighting of keywords with context to other text has been solved by calling highlighting function for variables with lexer rule id 9.
• Wrote a new test named syntaxtest to check the style of all possible tokens in the openscad script.

6 July

• A lexer rule for special variables in openscad starting with $ has been defined and highlighting function has been called for it.
• Also read about modifiers and tried to highlight them, but talked to mentor about its highlighting style as it should be highlighted just as a single character or as multiline comments.

7 July

• It is decided to highlight modifiers just as single character until the multiline comment problem is solved.

• Trying to read the documentation of scintilla http://scintilla.org/ScintillaDoc.html to find something useful for multiline comment highlighting successfully.

8 July

• Read Qscintilla existing lexer such as LexMSSQL to get the idea of how these lexers are handling multiline comments.
• Also get the concept of multi state for handling multi line comments.

9 July

• SCI_GETSTYLEAT function has been found to get the style at a particular position.
• SCI_GETCURRENTPOS function will be using to get the location of cursor.
• Wrote code in the scadlexer to get the style at cursor position whenever user make any change in the script.

10 July

• Used debugger to solve the problem of highlighting function being called again and again for some tokens. The problems was missing break statement in switch statement in some cases.

• Crashing of the application solved by removing additional setStyling statement in highlighting function and put that in if condition.

11 July

• Used multi state feature of lexertl and changed the state of statement having existing style as comment or style at starting location as comment to comment with statement "results.start = 1", means like telling the lexer to highlight a statement inside /* and */ as comment and not as regular text and keywords.
• With this the problem of multiline comments has been solved.
• Moving forward to apply same feature for highlighting modifiers.

13 July

• Reading about different modifiers, so that they can be styled, obvious to the regular user.
• Defining state for different modifiers *, %, !, # in lexer.

14 July

• Wrote lexer rules for all modifiers but it is not being highlighted as expected.

15 July

• Made the modifier rules working in the lexer.
• Defined different rules for single line modifiers and block of modifiers.

16 July

• Added new values in enum of scadlexer.h for modifiers
• Also called different style for modifiers in scintillaeditor.cc

17 July

• Defined highlighting colors for modifiers in all color scheme of editor which are written in json format in colorschemes/editor directory.

18 July

• Started reading about adding options to change the toolbar icons by user.
• User may also be able to add his own iconset.