Macro to reformat files

Posted by michael.cartmell on 2007-02-15 17:29

I use code beautifiers to reformat my code & XML etc.

This macro

  1. selects the appropriate formatter and options for the language
  2. saves the file
  3. reformats the text
komodo.assertMacroVersion(2);
if (komodo.view.scintilla) { komodo.view.scintilla.focus(); } // bug 67103

var formatter;
var language = (komodo.document || komodo.koDoc).language;
switch (language) {
    case 'Perl':
        formatter = 'perltidy';
        break;
    case 'XML':
    case 'XUL':
    case 'XLST':
        formatter = 'tidy -q -xml -i -w 80';
        break;
    case 'HTML':
        formatter = 'tidy -q -asxhtml -i -w 80';
        break;
    default:
        alert("I don't know how to tidy " + language);
        exit(1);
}

//save current cursor position
var currentPos = komodo.editor.currentPos;

try {
    // Save the file.  After the operation you can check what changes where made by
    // File -> Show Unsaved Changes
    komodo.doCommand('cmd_save');

    // Group operations into a single undo
    komodo.editor.beginUndoAction();

    // Select entire buffer & pipe it into formatter.
    komodo.doCommand('cmd_selectAll');
    Run_RunEncodedCommand(window, formatter + " {'insertOutput': True, 'operateOnSelection': True}");

     // Restore cursor.  It will be close to the where it started depending on how the text was modified.
     komodo.editor.gotoPos(currentPos);

    // On windows, when the output of a command is inserted into an edit buffer it has unix line ends.
    komodo.doCommand('cmd_cleanLineEndings');
}
catch (e) {
    alert(e);
}
finally {
    // Must end undo action or may corrupt edit buffer
    komodo.editor.endUndoAction();
}

It is easy enough to extend to other languages by adding extra cases. eg

    case 'C++':
        formatter = 'astyle --style=kr';
        break;
    case 'Java':
        formatter = 'astyle --style=java';
        break;

Michael Cartmell

jeff.griffiths | Fri, 2007-02-16 14:37

In order to answer a user's questions today I wrote a much shorter Macro today that only does perltidy:

var ke = komodo.editor
if (komodo.view.scintilla) { komodo.view.scintilla.focus(); }
var currentPos = komodo.editor.currentPos;

try {
    var tidycmd = "perltidy -i=2 -ce"
    komodo.doCommand('cmd_save');
    ke.beginUndoAction();
    komodo.doCommand('cmd_selectAll');
    Run_RunEncodedCommand(window, tidycmd + " {'insertOutput': True, 'operateOnSelection': True}");
    ke.gotoPos(currentPos);
    komodo.doCommand('cmd_cleanLineEndings');
} catch (e) {
    alert(e);
} finally {
    ke.endUndoAction();
}

--
JeffG | Komodo IDE 4.0 | MacBook Pro OS X Intel / Ubuntu 6.10 i386

wendymcf | Thu, 2008-08-28 08:11

This is just what I was looking for. And now I've also learned a thing or five about how to use Macros in Komodo. Thank you!

PHP
jeff.griffiths | Fri, 2007-02-16 14:51

I'm looking into figuring out the PHP_Beautifier pear package with a mind to integrate it into this Macro, but the docs are a bit dense. Has anyone used this, or know of a better one for PHP?

--
JeffG | Komodo IDE 4.0 | MacBook Pro OS X Intel / Ubuntu 6.10 i386

hefebia | Sat, 2007-02-17 17:13

I have used PHP Beautifier.

When you install php_beautifier via pear it installs a command line script that can be used similar to the other beautifiers:

'php_beautifier -s4 -l"Pear()"'

(or try php_beautifier --help)

However it seems that it only works on whole files. Sometimes it strips a few of the last characters in a file.

It uses stdin and stdout by default - so you can use the " {'insertOutput': True, 'operateOnSelection': True}" method.

doCommand('cmd_cleanLineEndings') is necessary, too.

davegaramond | Thu, 2007-06-14 16:17

Thanks for this very useful macro. I'm not well-versed enough in Komodo nor Javascript to modify this to only reformat current selection (instead of whole files). Anybody willing to help? Thanks in advance.

jonathaneunice | Sun, 2007-07-08 15:56

I'm extremely new to scripting Komodo, but it appears that if you prefix the komodo.doCommand('cmd_selectAll'); with the JavaScript // comment marker, the macro will reformat only the part of the file that is currently selected.

michael.cartmell | Sat, 2007-07-28 21:56

You can wrap cmd_sellectAll and gotoPos in if blocks so when there is a current selection that is passed to the reformatter otherwise the entire file is selected and passed. If text was selected then I want it to remain selected.

var text_not_selected = komodo.editor.selText == "";
if (text_not_selected) {
    komodo.doCommand('cmd_selectAll');
}
Run_RunEncodedCommand(window, formatter + " {'insertOutput': True, 'operateOnSelection': True}");

if (text_not_selected) {
    komodo.editor.gotoPos(currentPos);
}

Some formatters don't keep the initial indent, tidy, for example changes

<trunk>
  <branch>
    <twig>Longish text that will be wrapped to whatever column you set tidy
    to wrap to. It goes on for a bit and wraps multiple lines.</twig>
  </branch>
</trunk>

to

<trunk>
  <branch>
<twig>Longish text that will be wrapped to whatever column you set tidy to
wrap to. It goes on for a bit and wraps multiple lines.</twig>
  </branch>
</trunk>

if twig is selected. Simply outdenting the reformatted selection won't help as the lines are wrapped in the wrong place. One possibility is to have a guard variable set in the case statements and reformat the whole file if the guard is set.

var cannot_tidy_selection = false;
switch (language) {
    case 'Perl':
        formatter = 'perltidy';
        break;
    case 'XML':
    case 'XUL':
    case 'XLST':
        formatter = 'tidy -q -xml -i -w 80';
        cannot_tidy_selection = true;
        break;
...

var text_not_selected = cannot_tidy_selection
                     || komodo.editor.selText == "";
...

An alternative is to count the white space at the start of the selection, calculate the number of columns , subtract that from the number of columns in each line and then outdent the reformatted selection by that amount. An exercise I leave for others, you may find komodo.editor.tabWidth and komodo.editor.indentWidth useful;)

Michael

davegaramond | Thu, 2007-07-19 19:30

Thanks jonathaneunice, I've forgotten about this thread. Yes, that's exactly what I had done. And I also commented the cmd_save line, to make things a bit faster.

keanzu | Sat, 2007-07-21 00:59

I created an account here just to thank you for this macro. Works perfectly and does exactly what I want.

unixguy | Tue, 2007-07-31 15:27

I tried using this for the first time today. Worked fine with shorter perl scripts. But when I used it on a perl script with 270 lines, it cut off the last dozen characters or so. When I run the same script through perltidy from the shell, the formatted version is not truncated. So it would appear to be related to running perltidy from within komodo.

Any ideas?

jeff.griffiths | Tue, 2007-07-31 17:14

I generated a really large ( but fairly simple ) Perl file with 6000 lines and 1000 subroutines in it, and the Macro handled it fine. I'm using Komodo 4.2 Beta 5 and the latest perltidy installed from ppm, on an iMac.

Does this happen consistently for a specific file? If so, I'd love to look at that file. Email a zipped archive of it to support@activestate.com.

--
JeffG

unixguy | Tue, 2007-07-31 17:37

I'm using version 4.1.1, build 279677.

Don't know if I could sent the file due to confidentiality agreement. It happens consistently with this particular 270 or so line script.

I did a test on the file and added a blank line, plus:
# one
#two
#three
#four

to the end of the script. When I use the macro, It is cut off at
# one
#two

If I run the macro again, the remaining two added comment lines and the last line of the script are deleted.

The other anomaly, is that the point is moved up in the buffer from where it was before the macro was run. When I run the same script through perltidy at the shell, no truncation takes place.

I tried the macro on another 600 line script and there was no trucation.

unixguy | Tue, 2007-07-31 18:09

It appears to be cause by some commented out format statements! This trivial example shows what happens. Hopefully, you can replicate the problem.

This is the entire file before running the macro at the top of this page:

#!/usr/bin/perl -w
use strict;

#format STDOUT_TOP =
#                        @||||||||||||||||
#                        $server
#------------------------------------------------------------------
#.

#format STDOUT =
#@<<<<<<<<<<<<<<<<<<<<<<<<< @│││││││
#$signin,              $date
#.

 print "Help Desk -- What Editor do you use?";
 chomp($editor = <STDIN>);
 if ($editor =~ /emacs/i) {
   print "Why aren't you using vi?\n";
 } elsif ($editor =~ /vi/i) {
   print "Why aren't you using emacs?\n";
 } else {
   print "I think that's the problem\n";
 }

This it the result of running the script once:

#!/usr/bin/perl -w
use strict;

#format STDOUT_TOP =
#                        @||||||||||||||||
#                        $server
#------------------------------------------------------------------
#.

#format STDOUT =
#@<<<<<<<<<<<<<<<<<<<<<<<<< @│││││││
#$signin,              $date
#.

print "Help Desk -- What Editor do you use?";
chomp( $editor = <STDIN> );
if ( $editor =~ /emacs/i ) {
    print "Why aren't you using vi?\n";
}
elsif ( $editor =~ /vi/i ) {
    print "Why aren't you using emacs?\n";
}
else {
    print "I think that's the p

When run a second time, it continues to trucate, etc.

#!/usr/bin/perl -w
use strict;

#format STDOUT_TOP =
#                        @||||||||||||||||
#                        $server
#------------------------------------------------------------------
#.

#format STDOUT =
#@<<<<<<<<<<<<<<<<<<<<<<<<< @│││││││
#$signin,              $date
#.

print "Help Desk -- What Editor do you use?";
chomp( $editor = <STDIN> );
if ( $editor =~ /emacs/i ) {
    print "Why aren't you using vi?\n";
}
elsif ( $editor =~ /vi/i ) {
    print "Why aren't you using emacs?\n";
}
else {
    print "I think

jeff.griffiths | Wed, 2007-08-01 00:08

Thanks for digging into this - I can reproduce the problem now and will log a bug tomorrow.

--
JeffG

jeff.griffiths | Wed, 2007-08-01 13:21

@yaconsult: We are now tracking what seems to be a truncation issue related to multi-byte characters in this bug report:

http://bugs.activestate.com/show_bug.cgi?id=71441

Thanks again for the help.

--
JeffG

unixguy | Wed, 2007-08-01 15:24

Jeff,

You're welcome. It's a pretty odd bug.

But since the trivial example above that exercises the bug has no multi-byte characters in it, could it still be multi-byte related? It seems that just having the commented out format statements will cause the truncation.

ericp
ActiveState Staff
Wed, 2007-08-01 17:02

Some of those bar characters in the format statement weren't ascii
pipe/bar chars (chr 124). They were actually Unicode 0x2503's -- this
makes total sense, as the 0x25xy (x in 0..7, y in 0..f) characters
are for drawing boxes.

GroessterNehmer | Mon, 2008-07-07 15:22

I tried your macro but on Mac OS (at least for me) it totally screwed up any non-ASCII characters. I tried different encodings in Komodo (Edit - Current File Settings), mainly Latin1 and Mac Roman, but the result was always the same. For example Jägermeister becomes J& Atilde;& curren;germeister (I inserted spaces so the browser won't display the HTML entities). However, if I save the file and use tidy on the command line with your settings (tidy -q -asxhtml -i -w 80) everything is peachy. I noticed that every non-ASCII character gets translated into two HTML entities. If I had to guess I would say that it has something to do with how Komodo on Mac OS (or Mac OS itself) handles the clipboard. Is there a workaround for that? Could you modify the macro to save the file, use tidy on it and then reload it? I tried to find out how this could be done but this macro language is all gibberish to me.

Anyways, I find it somewhat strange that Komodo has no build in HTML entities converter which makes it practically useless for any HTML editing in languages that use non-ASCII characters (there are a lot). This is sad because otherwise it's a really nice program.

Just had an idea: could someone point me in the right direction on how to write a macro which finds single characters and replaces them with strings? Then I could replace the most common german non-ASCII characters with their HTML-codes, which would be good enough for me.

Florent V. | Wed, 2008-07-09 02:14

Anyways, I find it somewhat strange that Komodo has no build in HTML entities converter which makes it practically useless for any HTML editing in languages that use non-ASCII characters (there are a lot).

I disagree. I edit files in French, and I barely use HTML entities at all. I mean, the use of HTML entities is limited when you can just use ISO-8859-1 and UTF-8. I guess if I was japanese I'd use UTF-8, ISO-2022-JP or Shift-JIS.

HTML entities are only useful if you have absolutely no control of what charset will be used on the website. Like when you're creating a CMS theme that you intend to distribute, for a CMS which doesn't use UTF-8 as a standard.

But, for those very few times when you need to use HTML entities, you can just use Code » Convert Selected Text » Encode HTML Entities, which works fine (though it uses numeric HTML entities). I'm not sure whether this function is in the standard Komodo package. It might be a function from the MoreKomodo extension.

toddw
ActiveState Staff
Wed, 2008-07-09 09:42

The "Code » Convert Selected Text »" menu commands are provided by the MoreKomodo extension, not stock Komodo.

GroessterNehmer | Thu, 2008-07-10 05:21

I know that you can specify a different charset like UTF-8 but for compatibility reasons I always use HTML entities. And sometimes you can't specify the charset when you can only edit part of the source code (like on ebay). If you use for example UTF-8 for all your HTML pages I'm quite sure that at least some visitors will see strange symbols instead of your intended characters. I agree that it shouldn't be that way and browsers should show content like it is specified in the standard but some don't. I just want to be on the safe side. When I build some parts of a website for a company I had to write code like "if browser is IE do this, for opera do that and for firefox do it that way" just because each browser showed the page differently. That is not the way it was intended to be but you got to work with what you got. If your boss (or customer) says it doesn't look like it is supposed to be you can't just say "then get a decent browser".

Just my two (euro)cents.

Florent V. | Thu, 2008-07-10 06:13

If you use for example UTF-8 for all your HTML pages I'm quite sure that at least some visitors will see strange symbols instead of your intended characters.

If you use UTF-8, declare UTF-8, and don't mess up your UTF-8 content with old PHP functions that have trouble handling it, you shouldn't have this problem. Visitors will only see strange symbols if their system fonts don't have some of the signs you used in your content. But this problem will be exactly the same if you use HTML entities instead.

So it all boils down to using the right charset, declaring the right charset, and using a language (or a subset of this language) which is able to handle the chosen charset. There is no support problem for UTF-8 in “modern” browsers (starting with the IE4/NN4 generation... and these are not even considered modern right now, they're considered extinct. So you won't have to tell your boss, client or customers to get a decent browsers. All current browsers are decent regarding charsets.

By the way, unless I'm mistaken, ebay.com uses UTF-8 throughout. Wrong example. ;)

znmeb | Tue, 2008-07-08 20:45

Are there any C code beautifiers for Komodo? I've inherited a few hundred lines of C code that is an unholy mess -- almost unreadable. Is there something I can get that works with Komodo to clean up the indentation, etc.?

I opened the files up in Komodo Edit 4.4 this morning and I didn't see any obvious way to do this.

michael.cartmell | Wed, 2008-07-09 03:07

I use astyle. It does C, C++, C# and java.

ericp
ActiveState Staff
Wed, 2008-07-09 09:45

... which you'll find on most Linux and OS X systems (probably
as part of XCode there). It's showing its age these days --
it modifies its input file it you don't provide an output file
argument, and uses old-school options like "-bap" (for blank
line after every procedure body).

I have a personal aversion to any program that modifies
its input file by default, but as a Komodo run-command
it might make sense. If you don't like the results, you
can undo them.

- Eric

znmeb | Wed, 2008-07-09 19:33

Well, I tried astyle (on a Windows system). The source file looked a "little better", but it was still basically unreadable, because it's poorly factored. So I think I'm going to need to bite the bullet and force the programmers to write unit tests, so I can clean the code up.

Florent V. | Thu, 2008-07-10 02:08

Hello,

Unless I'm mistaken, the formatter command line for HTML (using Tidy) will destroy data if you're not using plain ASCII text.

It uses the default values for character encoding, which means that:
- input is seen as ISO-8859-1;
- every byte that is out of the ASCII range will be converted to a HTML entity;
- output is plain ASCII.

So your ISO-8859-1 (latin1) characters that are not ASCII-compatible will be converted to HTML entities whether you want it or not, and if you use another charset (such as UTF-8) you will get garbled data.

This can be fixed using the charset options of Tidy, but you have to tell it what charset the file is.

I tried to do that but I found no way to get the current document's charset while searching the Komodo Macro API. Is there an undocumented feature, or is that a missing feature?

toddw
ActiveState Staff
Thu, 2008-07-10 14:54

I think the best option here is always specify the utf8 charset with tidy when operating on a Komodo editor selection. UTF-8 is the encoding that is used for the "operateOnSelection" (as internally, Komodo is always using UTF-8 encoding, which is later converted back to the original document's encoding upon saving).

The encoding setting is a part on the koIDocument (XPCOM) interface, i.e.:

ko.views.manager.currentView.document.encoding

kaapstorm | Fri, 2010-02-12 00:45

This is fantastic! Thank you to everyone who has contributed to this topic.

I've collated all the suggested changes into one chunk. I've posted them on my blog (linking to this page, and giving credit, of course). I'd add the URL here, but I'm triggering the spam filter. If you're interested, the post is at www dot digitaltoolcompany dot com slash blog slash archives slash 149.

Here's the collated code (including commented-out bits):

komodo.assertMacroVersion(2);
if (komodo.view.scintilla) { komodo.view.scintilla.focus(); } // bug 67103

var formatter;
var language = komodo.document.language;
var cannot_tidy_selection = false;
switch (language) {
    case 'C':
        formatter = 'astyle --style=linux';
        break;
    case 'C++':
        formatter = 'astyle --style=kr';
        break;
    case 'C#':
        formatter = 'astyle --style=ansi';
        break;
    case 'HTML':
        cannot_tidy_selection = true;
        formatter = 'tidy -q -utf8 -asxhtml -i -w 80';
        break;
    case 'Java':
        formatter = 'astyle --style=java';
        break;
    case 'Perl':
        formatter = 'perltidy';
        break;
    case 'PHP':
        formatter = 'php_beautifier -s4 -l"Pear()"';
        break;
    case 'XLST':
    case 'XML':
    case 'XUL':
        cannot_tidy_selection = true;
        formatter = 'tidy -q -utf8 -xml -i -w 80';
        break;
    default:
        alert("I don't know how to tidy " + language);
        exit(1);
}

//save current cursor position
var currentPos = komodo.editor.currentPos;

try {
    // Save the file.  After the operation you can check what changes where made by
    // File -> Show Unsaved Changes
    komodo.doCommand('cmd_save');

    // Group operations into a single undo
    komodo.editor.beginUndoAction();

    // Select entire buffer & pipe it into formatter.
    //komodo.doCommand('cmd_selectAll');
    //Run_RunEncodedCommand(window, formatter + " {'insertOutput': True, 'operateOnSelection': True}");

    //var text_not_selected = komodo.editor.selText == "";
    var text_not_selected = cannot_tidy_selection
                     || komodo.editor.selText == "";
    if (text_not_selected) {
        komodo.doCommand('cmd_selectAll');
    }
    Run_RunEncodedCommand(window, formatter + " {'insertOutput': True, 'operateOnSelection': True}");

    if (text_not_selected) {
        komodo.editor.gotoPos(currentPos);
    }

     // Restore cursor.  It will be close to the where it started depending on how the text was modified.
     komodo.editor.gotoPos(currentPos);

    // On windows, when the output of a command is inserted into an edit buffer it has unix line ends.
    komodo.doCommand('cmd_cleanLineEndings');

} catch (e) {
    alert(e);

} finally {
    // Must end undo action or may corrupt edit buffer
    komodo.editor.endUndoAction();
}

css
oozypal | Tue, 2010-04-20 10:36

Hello,

How about css. I have csstidy install on Ubuntu.

How can I add it to the JS.

Thx

oozypal | Tue, 2010-04-20 12:00

Thank you, I found the answer somewhere else

    case 'CSS':
        formatter = 'csstidy - --preserve_css=true --lowercase_s=true --case_properties=true --sort_properties=true --template=low --remove_bslash=false';
        break;

designermonkey | Tue, 2011-01-18 07:30

Hi does anyone know how to update this for Komodo Edit 6?

I get a warning to change komodo.assertMacroVersion(2); to komodo.assertMacroVersion(3); but then it fails to work.

I don't know enough about the inner workings of Komodo to start playing myself.

Cheers in advance...

ericp
ActiveState Staff
Tue, 2011-01-18 11:28

It gives a warning about the macro version, but then continues
to do its thing, and the selected code is reformatted.

On one run I was told to see the file "perltidy.ERR", but I
can't find it in the directory the source file is in.

- Eric