Uploader

Posted by machine on 2010-05-20 15:58 |
1.62
Adam Lyskawa
Fri, 2013-02-22 06:36
  • : 4.1 - 8.*
  • : 4.1 - 8.*

Project based automatic file upload feature

Komodo already has upload feature, but it's not very usable, because it takes a lot of clicks to actually upload a file to remote server. You'd have to choose server, and then target path each time. It misses really quick solution, where project's upload settings are preconfigured, and all you need is actually commit file, with no questions asked.

Uploader makes web programmer's life brighter. First, if you assign a keyboard shortcut to "General: Upload" command, you can commit project files immediately. Then, if you configure your project for automatic upload, project files will be uploaded on each save action, exactly like in NetBeans.

Configuration:

  • go to Edit / Preferences / Server
  • add new remote account with a name matching the name of your project
  • precede the name with "*" to enable Auto Upload feature for this project
  • that's it!

What do you think about new notifications?
Should they stay or should they go?

New in version 1.62:

  • FIXED: Komodo 8.x compatibility issues.
  • CHANGED: Internal dependencies.

New in version 1.61:

  • CHANGED: Komodo 7.x notifications.

New in version 1.60:

  • FIXED: oops, "ain't work AGAIN!" bug.

New in version 1.59:

  • FIXED: oops, "ain't work" bug.

New in version 1.58: (unreleased)

  • NEW: new extension isolation model used.

New in version 1.57:

  • Komodo 7 support!

saren | Sun, 2010-05-30 21:00

Thanks for this great extension. The only problem I have is that the passwords need to be re-entered every time I restart the program. They are saving in the servers section of preferences and opening remote files works without password prompts after restarting but not for uploader. I'm using ubuntu 10.04.

machine | Sun, 2010-05-30 21:21

Have you tried using SCP and SFTP?

If so, and it didn't help, there's a generic solution for paswordless authentication. See http://www.linuxjournal.com/article/8600. I'm using it with my Ubuntu 10.04.

I haven't tested it with FTP yet, what type of upload do you use?

saren | Mon, 2010-05-31 11:24

No I haven't tried SCP or SFTP. Thanks for the suggestion I will check it out.

Came across one other problem, when a server has no Default Path Uploader displays the "No configuration found." message. I tried putting in a / but then Uploader tried using // which doesn't work.

machine | Mon, 2010-05-31 11:47

Well, I think you've found a bug then. It's the message "No configuration found" - it could happen only if path was empty, but when theres "/" it's definitely not empty. I'll try to fix it ASAP and test it with FTP account. Version 1.0.1 should be available later today.

saren | Mon, 2010-05-31 13:58

Sorry but I got that message when there was no path - there was no "/" in the default path field. If I put a "/" in there Uploader chokes because it tries to open that path as "//"

Hope that helps.

machine | Mon, 2010-05-31 14:51

I tried to test it with newly created FTP account, but they banned me for testing, some really bad hostings have IP bans for several bad logins. With another account now it seems to work. You can provide password in server configuration and it's used, or leave it blank - Komodo will ask for it just once per session.

I fixed bug with double slash, though it didn't prevent uploading to my test site, but maybe some servers don't like double slashes. Anyway - now it should work :)

Yay
saren | Mon, 2010-05-31 20:08

The new version works perfectly. Thanks so much!

vipx1 | Sat, 2010-06-05 08:25

SSH has a FTP function which is secure, SFTP.
If have SSH running on the server then you can use Ubuntu's Password and Key wallet or Putty on Windows to manage the server key and you will not be asked for password when you connect if you use a key with an empty pass-phrase.
Also your data is encrypted. The connection is only possible with the key if you switch of password authentication on the SSH server.
If you set up a server (no password) in komodo and open the file from the server listing (remote file) then when the file is saved it is uploaded onto the server (upload). ServerTree add-on helps with this process.

machine | Sun, 2010-06-06 21:25

I've just fixed a load of bugs from 1.0.x versions, so please check the new version out.

murphy42 | Thu, 2010-06-24 05:20

Uploader version 1.1,
Komodo IDE, version 5.2.4, build 37659, platform win32-x86.

The local path of my project is appended to the "Default path" which is set in "preferences->servers->remote accounts"

[ 2010-05-04 15:12:21 ] Active project: ispdd, UPLOADING: test.php => /C:/Program Files/BitNami WAPPStack/apache2/htdocs/ispdd/projects/test.php... FAILED.
[ 2010-05-04 15:12:30 ] Active project: ispdd, NOTICE: Check your connection settings and / or remote permissions.

machine | Thu, 2010-06-24 11:07

It seems like your project is on the local machine, so why bother uploading? Is your Windows server configured for FTP/SFTP? If so, you should enter the path from which your project is visible via FTP/SFTP.

As "Default path" you should enter remote path, if the server is local, the path should be as in any FTP/SFTP client. I haven't tested it with Windows servers, so tell me which FTP/SFTP server do you use, I'll install and configure it, and then I'll be able to test it.

It seems quite weird the local path is appended to your remote path, I cannot reproduce it. The remote URI is built from the local file path with the path of the project file (.kpf|.komodoproject) cut. If the project file is not in the same directory as project files (for example on another partition) - this could be the case. Just try to create new project file in the root directory of your project - that should work.

Again it's weird, because if I try to upload file not from the project file directory I get a message "the current file is unrelated", not "error uploading..." (just as it should be when you explicitly try to upload unrelated file)

murphy42 | Thu, 2010-06-24 22:29

The project is on my local machine, in my local test server root.
And I "bother" uploading it to a remote, production site.

The .kpf file is in the project root, i already tried deleting and recreating the project, no luck. Although "Delete project" doesn't work from inside Komodo, i deleted the project file manually.
Seems to me..

The remote URI is built from the local file path with the path of the project file (.kpf|.komodoproject) cut.

that this doesn't happen. The path of the project file isn't "cut".

I'll try reinstalling the whole Komodo IDE, will post later.

murphy42 | Thu, 2010-06-24 22:52

Fresh Install,
Komodo IDE, version 5.2.4, build 37659, platform win32-x86.
Uploader 1.1
Vista Home Premium SP2, x86

Project file is where it should be and remote site is set up as it should be. Still doesn't work, same mistake

[ 2010-05-05 08:45:03 ] Active project: ispdd, UPLOADING: test.php => /C:/Program Files/BitNami WAPPStack/apache2/htdocs/ispdd/projects/test.php... FAILED.
[ 2010-05-05 08:45:06 ] Active project: ispdd, NOTICE: Check your connection settings and / or remote permissions.

regards,
Murphy

murphy42 | Fri, 2010-06-25 01:46

The problem is whitespace

<strong>localPathPrefix: </strong>C:/Program%20Files/BitNami%20WAPPStack/apache2/htdocs/ispdd
<strong>localPath: </strong>C:/Program Files/BitNami WAPPStack/apache2/htdocs/ispdd/projects/test.php

localPathPrefix comes from this.project.url
and localPath comes from this.document.file.path

from which you get "urlencoded" whitespace and "plain" whitespace;

For a quick and dirty solution..
this works
this.localPath = this.localPath.replace(/\s/g,"%20");

The files do get uploaded now, but I don't get any console notifications for success which I would like.
Not much time to look into it further though...

regards,
Murphy

machine | Fri, 2010-06-25 08:49

You've found a huge bug, shame on me, you've pinpointed it! It was a late night, or early day here, so I was offline. I'll fix it ASAP. Thanks for your help. I'll check those notifications too.

...

Done. Tested. Seems to work now.
Changed line 107 of uploader.js to:

this.localPathPrefix = decodeURI(this.project.url).replace('file://', '').replace('/' + this.project.name, '').replace(/^\/([A-Z]:)/, '$1');

and line 113 of servers.js to:

if (remotePath) uri+=  remotePath.replace(/ /g, '%20');

It seems like it's no cleaner way to do so. The project url property is URI-encoded inside Komodo itself and paths read from other properties are not. In the second change I don't really know why encodeURIComponent() doesn't work. It encodes slashes which Komodo does not decode back like spaces. Doh. But it works now :) I tried to use unicode in paths, but Komodo refused to save changes. But it's just Komodo bug :)

Please let me know if the updated version works for you.

Best regards,
Adam

machine | Fri, 2010-06-25 08:52

FIXED: bug with project paths containing spaces (thanks to murphy42!)

murphy42 | Fri, 2010-06-25 12:18

Confirmed, working.

and thanx for this essential feature that Komodo was missing

regards,
Murphy

slvrspiral | Sat, 2010-07-10 20:27

If you try to upload a file to a directory on your side that doesn't exist yet server side, you have to use another program to create the directory then upload your file. Most other tools that do the same will just create the directory then upload the file. It would be awesome if this did the same.

machine | Sun, 2010-07-11 17:31

Check the new version out, it does it :) It was a tricky thing to do.

yeurus | Sun, 2010-07-25 06:52

My komodo edit version is 6.0 beta 2.
if i configured server path like "path/",uploader plugin won't works,
i have to add project name in front of it to make it works,like "projectname/path/"
but other plugins will works with automatic added project name at front.
i already fixed this bug in uploader.js line 120~122:

	this.remoteDir =this.project.name+this.remotePath
	  ? this.project.name+this.remotePath.replace(/\/[^\/]+$/, '')
	  : ''; 

now it works fine,but i don't think this is the best solution,hope you can fix it on following version.
thanks

machine | Sun, 2010-07-25 07:08

What plugins add project name to remote path? I agree adding project's name to remote path would be good for simplicity, but think about flexibility. What if someone wants to upload the project into different directory? I sometimes add "qa." before the directory name for testing versions, no prefix for production ones.

yeurus | Sun, 2010-07-25 18:45

Remote drive tree will add project name to remote path,and also jsTreeDrive,
maybe you are right,i think can add an option to let user choose?
actually i don't mind to use same solution on every updates,
that's mean i don't mind if you ignore this "bug" :)

cdumay | Wed, 2010-08-11 03:11

In the server configuration you can specify alternate port to use but it seems that this add-on doesn't use it.

[2010-08-11 11:54:21,041] [ERROR] koSFTPConnection: SFTP do_openSocket ERROR: Connection refused
[2010-08-11 11:54:21,044] [ERROR] koDocument: save: can't write to file [...]

NB: The server configuration works with the Remote Drive Tree extension.
NB2: The remote path displayed in debug is correct and permissions are right.

machine | Sat, 2010-10-09 07:46

I've changed the function which in fact ignored port configuration, but I was unable to test it, because though upload URI is created properly now, it still fails to fail when I enter a random port number. Maybe Komodo just uses a fallback when configured port is unavailable, I don't know, try 1.51 and tell me.

cdumay | Fri, 2010-10-22 03:11

no it doesn't works on 1.55

take a look to http://88.191.76.164/komodo.mpeg

machine | Fri, 2010-10-22 04:22

In 1.56 I'll do my best to fix this. If I had linux installed it would be easier to test, but i'll try to configure it somehow on company server, I hope I won't break anything :)

There is another very serious bug in uploader since the first version - when more than one Komodo window is opened, it tries to perform upload from active project in inactive one.

So I try to fix both those bugs today.

machine | Thu, 2010-10-28 16:03

It seems like a bug in Komodo itself... [EDIT]: (yada yada yada...)

Nope. Shame on me. My silly bug, almost a typo in URI. Todd has helped me to find it so since 1.56 Uploader should work with non-default remote ports too.

cdumay | Tue, 2010-11-02 03:12

Tested with Komodo 6.0.1 and Uploader 1.56, it works with an alternate port

Good job!

mcdope | Fri, 2010-10-08 03:14

For me Uploader seems to don't work at all...

It wont upload, no error messages are shown... Is there a log somewhere?

machine | Fri, 2010-10-08 13:55

I haven't tested it with IDE yet, I'll try to check this out today. Sorry, no log, even if it was one, no error shown means the event is not hooked or worse. I'll check this out (and the port bug too).

UPDATE:
I've just tested it with shiny new official Komodo IDE 6.0 - it seems to work, so let's pinpoint possible problems:

1. Server configuration from existing Komodo Edit installation is not imported into Komodo IDE 6, you have to manualy setup your server, please notice that automatic upload feature DOES NOT DISPLAY ANY MESSAGES if not configured, and it's done on purpose: if you just have the add-on installed but not currently using it - it should not interfere in any way with normal Komodo operation. It's designed to exit quietly as soon as possible when no configuration is found for the current project. If you use manual upload feature (accessible from file menu) it should display an error message - "NO CONFIGURATION FOUND". To make it work just check the Servers options out, set the server name to project name (AFAIR it's case sensitive), preceed it with '*' to activate automatic upload. Now, even if your configuration still has an error in it (like wrong remote path or password) - an error message should be displayed on save.

2. There's a highly unlikely that an other add-on conflicts with Uploader, try to disable other plugins and see what happens.

3. It's even more unlikely that server you're trying to connect returns so unexpected response, that either the add-on or Komodo itself doesn't know how to handle it. Check if you can connect with Komodo alone, without using the add-on. You don't need separate configuration for it, the one you set up for your project is enough.

4. You haven't opened / created a project? The file you're trying to upload is not from the current project? Uploader is project based. Active project is necessary for it to find the correct upload configuration. It's made this way to avoid separate configuration for the add on, while there is a great tool provided by Komodo by default. It has one more advantage: let's say we work on a project, but at the same time we open an add-on source to fix it. We can safely save it without being uploaded.

I've checked it only with Windows version of Komodo IDE, I don't have my Linux installed for now (I wait for new Ubuntu :) ) The OS things should cause no issues however, they were tested and fixed in previous versions.

OK, the port thing... I'm working on it right now.

mcdope | Sun, 2010-10-10 01:58

I double-checked my config now, it works for sure. With Komodos BuiltIn "Save as remote file" at least.

If i use File -> Upload it spawns an error message in Command Output that no configuration was found. Could it be that Uploader doesn't like Project names with spaces in them?

See this screenshot -> Click me

machine | Sun, 2010-10-10 14:19

It helps a lot, as soon as I walk my dog I won't rest until I'll make this thing work. Spaces, yes, could be, I should always test pathnames for spaces, their encoding is OS and protocol dependent. My mistake. Update in ca 2h.

OOPS. Problem. I stuck with my other, new add-on, NST (New Source Tree)... I guess I have to finish it or die. I'm on a hacking spree.

mcdope | Mon, 2010-10-11 11:38

I just got it working. I've renamed the project file, changed a reference inside it to the new name and needed to delete and re-add the server config. But now it's working.

But since spaces are not that uncommon a fix would be nice anyway :D

EDIT: Doesn't work, again -.- I give up and stick with manual upload till you fix it :P
EDIT2: Tried to remove all special chars from the project, still wont work... Argh...
EDIT3: Lol, got it working again - needed to add a "/" as "remote path" in the config

machine | Tue, 2010-10-12 08:05

Please help, I don't really know what's wrong with it, I've created a long directory name using spaces and "-". I put my project there, and named it with even longer name containing spaces, dashes and numbers. I configured automatic upload and it works from the start.

Tested with Komodo Edit 6.0, stable, Windows 7, SFTP. I don't have FTP server configured right now, but I tried to switch to FTP anyway, to see it fails. The timeout is slightly exceeded, but it displays nice error message.

So if there is a bug, please help, I don't know how to reproduce it.

mcdope | Tue, 2010-10-12 13:14

I just had a project with a space in the name... But i think the real source is somewhere else... Since it works since i added "/" as remotepath (i use a dedicated ftp user, which home is the remote project folder) i think maybe komodo reports the config as invalid without a remote path, which again uploader interprets as "no config"?

machine | Tue, 2010-10-12 15:37

Remote path had to be set. I've changed it in 1.52. Please check this out and tell if it helped.

mcdope | Tue, 2010-10-12 21:05

... without remote path now too. Nice =)

yermak99 | Sat, 2010-10-16 09:42

I’m very sorry to be a bit off topic. The Uploader is exactly what I need but before I get into that I need help with setting up Komodo + SSH public key authentication. The development machine is a Windows workstation the remote machine is Ubuntu box. I’ve setup PuTTY and pageant and that part works just fine. Now the Komodo part, I’ve been through the help and googled the setup procedure but nothing satisfactory surfaced. There is some info on setting up SVN over SSH, but I wasn’t able apply this info to my situation.

So if someone can point me in the right direction it would be greatly appreciated. The end goal is to get the Uploader working, but I’m stuck with setting up Komodo and SSH public key.

Thank you,
Alex

machine | Sat, 2010-10-16 11:02

I used the public key authentication with PuTTY and CygWin, they both worked after spending a lot of time on reading manuals and experimenting, but I used this to avoid entering password each time I connected target machine.

In Komodo you can just store the password and it's done. Of course, the password is only as safe there, as your Windows machine is. Anyway - if your machine is hacked, the key is even more valuable to the attacker than a stored (encrypted) password.

And yes, I really don't know how to use a public key in Komodo. Maybe try Komodo community forums?

yermak99 | Sat, 2010-10-16 21:08

Not sure what I was doing wrong before, but all of a sudden the Komodo + SSH public key authentication started to work. Installed your Uploader and ran into a couple of minor issues. Since server configuration lacks password in my setup the getURI routine in servers.js generates invalid URI, I’ve changed:

if (this.protocol && this.username && this.password) uri+= '@';
to
if (this.protocol && this.username) uri+= '@';

And if there is permissions issue the upload routine in uploader.js goes into infinite loop, so I’ve change it to break out of the loop after few re-tries.

But the bottom line your Uploader works absolutely great. Thank you very much for the excellent add-on.

Alex

machine | Sun, 2010-10-17 01:22

You've contributed to a better Komodo! All bugs fixed in 1.54.

leftjustified | Sun, 2010-10-17 11:48

Maybe I am stupid, but the uploader doesn't work at all for me.

I use "Komodo IDE, version 6.0.0, build 57120, platform linux-libcpp6-x86_64" and installed the addon. After that, I created a project "abc" and than I created a server with the name "*abc".

Now I tried to upload a file by clicking on file->upload but nothing happend...

Whats the problem?

Best regards

machine | Sun, 2010-10-17 12:00

I modified one file without checking and of course there was bug in it.
It's fixed now.
No more bugs, I've just uploaded some files to test :)

leftjustified | Sun, 2010-10-17 12:06

Thank you very much!

Now, Komodo is perfect for me ;)

diggie | Wed, 2010-11-24 07:36

Hi machine, and thank you for making this addon. It works perfectly!

There is however one thing that is not working, outside of Uploader, but related: When you want a project to auto-upload, you have to add a * in front of the connection name. However, if you want to use that (same) connection in the Remote Path of the Publishing feature of Komodo IDE, that will list a long list of block device names from /sys/block, like this: sys/block/sda2/subsystem/sda2/subsystem/sda2/subsystem/sda2/subsystem/sda2/subsystem/sda2/subsystem/sda2/subsystem/sda2/subsystem/sda2/subsystem/sda2/subsystem/sda2/subsystem/sda2/subsystem/sda2/subsystem/sda2/subsystem/sda2/subsystem/sda2/subsystem/sda2/subsystem/sda1/subsystem/ram11/subsystem/sda1/removable.

I think the Publishing feature doesn't really understand a path like sftp://*hostname.tld/path/to/files/, to put it mildly :-)

So instead of a wildcard character I think a dot (.) or another harmless character in front of the connection name is a better idea.

Maybe for backward compatability, you can support both?

leoncito | Sat, 2011-01-01 19:16

Hi!

Great extension! However I have a problem with passwords. Some of my passwords contain '*' or other special characters. Please, could add an escape function for password strings.

Thank you!

twohawks | Sun, 2011-02-06 13:13

Hi, Thank you for this most needed plugin.

When I first installed it, anytime I uploaded the affects would be reported in the command output section. After the first session I used it this no longer is the case.

I tried reinstalling, but ever since the first session no command output.
Is there something I can do to 'turn on' this feature? It helps to know when a successful, or not, upload occured and where it went.

Thanks.

machine | Sun, 2011-02-06 13:25

Please try to make sure you have the right project selected. There was a major change in project handling in Komodo since version 6.1. Now when you have more than one project open (now you can), and you try to upload a file which belongs to other project - it won't work. With automatic upload (with * before server name) it will fail silently - file is saved without uploading, with manual option it should output a message. I have those problems all the time since 6.1 :)

But gues what: now it never happens to even try upload a file into wrong place. Project handling before 6.1 was just wrong.

So, no output - no upload. (BTW, are you sure the Output tab is active? It's not activated automaticaly in auto mode, it would be annoying.)

So, any of these is the case?

rmarriner | Mon, 2011-03-07 01:06

Hello, first off I thank you for your time on this project. However I would like to report what I believe is a bug.

When connecting to my SFTP server the Uploader plugin works great until the connection is idle for longer than about 30 seconds to a minute. Then when I try to save a file, instead of using the existing connection Komodo will create a second connection. The problem is that it will eventually create a third, fourth, etc, etc without disconnecting the previous connections. I then hit my max log in limit on the SFTP server and Komodo cannot save the file.

I ssh into the server console and show all sshd processes, sure enough, there are tons of different connections for the user I was connected with. When I close Komodo all the sshd connections go away.

I tried setting the server timeout in the settings to 3600 in Komodo but that doesn't change this behavior. I've also tried setting timeouts server side with no effect.

This issues appears in the Komodo 6.1.1 using the latest Uploader source on both Linux (Ubuntu 10.10) and Windows Vista. The server is running FreeBSD 7.3, with base ssh (OpenSSH 5.1p1) with MySecureShell-1.25 as an SFTP wrapper. Filezilla works correctly on the server.

Thanks you!

baikis | Tue, 2011-03-08 22:54

this is a great extension !!

one note: it changes console text color - after uploading a file I do not see the text. I use white text on black background - the output is invisible.