HTML, UDL and code intelligence

Posted by gravism on 2008-02-06 14:38

After nearly two days of UDL research, I compiled my first working extension for a new language. Its a template language with additional server side script language, similar to Cold Fusion or JSP. First step was simply to create a working multi-language extension with HTML, JS and CSS, based on the existing UDL files for each language.
Code coloring works fine, but code intelligence for HTML does not. :(
My settings are very similar to the RHTML extension which has working code intel.

How can I get code intelligence to work with HTML, JS and CSS in my language?

The language ist named "WebTML", here are settings:

koWebTML_UDL_Language.py:

# Komodo WebTML language service.

import logging
from koXMLLanguageBase import koHTMLLanguageBase

log = logging.getLogger("koWebTMLLanguage")
#log.setLevel(logging.DEBUG)

def registerLanguage(registry):
    log.debug("Registering language WebTML")
    registry.registerLanguage(KoWebTMLLanguage())

class KoWebTMLLanguage(koHTMLLanguageBase):
    name = "WebTML"
    lexresLangName = "WebTML"
    _reg_desc_ = "%s Language" % name
    _reg_contractid_ = "@activestate.com/koLanguage?language=%s;1" % name
    _reg_clsid_ = "e98f0b3c-d13c-4066-9c5d-5c38084a980d"
    defaultExtension = '.tml'
   
    lang_from_udl_family = {'CSL': 'JavaScript', 'M': 'HTML', 'CSS': 'CSS',
                            'SSL': 'WebTML'}

webtml-mainlex.udl:

language WebTML

include "html2js.udl"

include "html2css.udl"

include "css2html.udl"

include "js2html.udl"

include "html.udl"
include "csslex.udl"
include "jslex.udl"

The included UDL's are original from the sdk-folder.
I use Komodo IDE 4.2.1 with the "ludditelib"-bugfix 72556.

Would be great to get some advice!
Thx

trentm
ActiveState Staff
Thu, 2008-02-07 11:17

gravism: Yes, this looks like a bug in the codeintel system picking up on your new extension. I can reproduce the problem. I've started a bug for this:

http://bugs.activestate.com/show_bug.cgi?id=74719

and hope to look at it soon.

trentm
ActiveState Staff
Thu, 2008-02-07 14:55

gravism: This isn't actually a bug in the codeintel system. Instead (and I admit this isn't super easy stuff) you need to explicitly define codeintel-support for new languages via a "pylib/lang_webtml.py" file.

"koext startcodeintel LANG" will generate a starter stub for this. However, there is a chunk of work required on the stub to get it going for your case. Mostly you want a lang_webtml.py that is the same as the lang_html.py that is part of Komodo.

I've put together a starter lang_webtml.py that gets autocomplete working in HTML, JS and CSS content. See the second attachment to: http://bugs.activestate.com/show_bug.cgi?id=74719

Cheers,
Trent

gravism | Thu, 2008-02-07 15:39

Thanks a lot, it works :)

Now I will try to include some custom WebTML-syntax, hope this will work too. The challenge for Komodo will be that markup of HTML and WebTML can be nested inside single tags, such like this:

<div class="<tml:item name="myclass" />">...</div>

So the HTML syntax checker and the code coloring must be able to ignore the tml-tag (and the quotes inside). All IDE's I tried before failed with this, code coloring was broken etc.

If Komodo does the job, it will be the IDE of my choice! :)

trentm
ActiveState Staff
Thu, 2008-02-07 15:53

Eric's the (UDL-)expert on whether Komodo's syntax coloring will be able to handle that.

ericp
ActiveState Staff
Thu, 2008-02-07 17:12

You can easily write a code-colorizer that knows how to jump from
HTML-mode to WTML-mode on a string like "". It won't even have problems figuring
out which quotes belong where. That's easy.

It also looks like the HTML syntax checker we use, htmltidy,
has no problem with "<" and ">" characters inside attribute
strings. This is not valid XML, though.

gravism | Tue, 2008-02-12 16:29

Ok, I moved a small step forward: The tml-tags are separated from the html-code, with this few lines of UDL:

state IN_M_DEFAULT: # HTML to WebTML
'&lt;tml:' : paint(upto, M_DEFAULT), paint(include, TPL_OPERATOR), spush_check(IN_M_DEFAULT), =&gt; IN_TPL_DEFAULT
state IN_TPL_DEFAULT:
/>/ : paint(upto, TPL_DEFAULT), paint(include, TPL_OPERATOR), spop_check, =&gt; IN_M_DEFAULT

Quite simple and not even smart.

But following things do not work (as I expected):

  • Each tml-tag has a green curly underline with an error that says <tml:whatever> is not recognized
  • Code intel and auto completion like in HTML: How do I get this to work
    in WebTML? Where do I define a set of valid tags and attributes?

What is the difference between code intelligence and autocompletion/calltips?

trentm
ActiveState Staff
Tue, 2008-02-12 16:46

> What is the difference between code intelligence and
> autocompletion/calltips?

Autocomplete and calltips are the features. "Code intelligence" (or codeintel for short) is the name of the Komodo sub-system that provides autocomplete and calltips for number of languages (as well as some other features like "Go To Definition" and the Code Browser).

Those "<tml:foo" things (I'm guessing here because I don't have a like to a WebTML spec :) are being checked by the XML/HTML syntax checker. And the HTML syntax checker doesn't like it them. For proper XML syntax checking you'd need to add an XML namespace declaration that defined the "tml" namespace.

Then for Komodo XML autocomplete you'd be able to add a schema (in DTD or XML Scheme or Relax NG format) in Komodo's "XML Catalogs" prefs panel.

gravism | Tue, 2008-02-12 17:56

Sorry, I don't get it. How do I make use of those autocomplete and calltips features? Are there any examples?

toddw
ActiveState Staff
Tue, 2008-02-12 18:41

The Komodo OpenSocial extension is one example that may be similar to what you are doing.

An OpenSocial xml document contains specific Google gadget xml tags, then inside of the content tag, it contains one large CDATA section containing normal html tags. To get the xml completions to work correctly between the two different sections, Shane had to do some specific hacking in the "pylib/lang_opensocial.py" file.

I'd imagine you'd need something similar to enable the "<tml:" completions.

Note: these specific xml_tree_handler changes in the OpenSocial "pylib/lang_opensocial.py" file need the latest Komodo 4.3.0 alpha/beta builds to work correctly. The difference for this particular extension is that for Komodo 4.2.1, when inside the cdata html section, only the gadget xml completions would get shown.

Cheers,
Todd

trentm
ActiveState Staff
Tue, 2008-02-12 18:19

corrected my previous comment that resulted in half of it now showing

gravism | Wed, 2008-02-13 16:11

The OpenSocial example seems to use strict separation between HTML and XML, which WebTML does not - it's not valid XML. I looked at this XML Catalog thing, but thats too heavy stuff for me at the moment ;)

However, I found a simple solution to get rid of the HTML-Tidy error messages: Teaching Tidy about new tags! is the key - no auto completion, but auto tag-closing at least! :)

For now, this solution works for me, the rest can be done by toolbox snippets.

Finally something for the wishlist:

  • Different coloring for different tags (wished my tml-tags look different than the other html-code)
  • Has nothing to do with custom languages, but: Why, for example, does Javascript code inside HTML-script-tag has different color coding than Javascript code in separate .js-files? Javascript coloring should be the same, no matter where the code is placed.