Converting HTTP to FTP URL

classic Classic list List threaded Threaded
16 messages Options
Reply | Threaded
Open this post in threaded view
|

Converting HTTP to FTP URL

J. Landman Gay
I need a way to convert a web URL to a valid FTP URL. For example,
convert this:

   http://www.domain.com/folder/file.txt

to this:

   ftp://user:[hidden email]/webroot/folder/file.txt

Not all servers use "public_html" so I can't just hard-code that as the
web root. Is there a generic way to indicate the web folder on a server,
the same way a tilde indicates the user's home folder? Or any other way
to get the path?

The user name and password are known.

--
Jacqueline Landman Gay         |     [hidden email]
HyperActive Software           |     http://www.hyperactivesw.com

_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

Re: Converting HTTP to FTP URL

J. Landman Gay
On 8/7/11 5:12 PM, J. Landman Gay wrote:

> I need a way to convert a web URL to a valid FTP URL. For example,
> convert this:
>
> http://www.domain.com/folder/file.txt
>
> to this:
>
> ftp://user:[hidden email]/webroot/folder/file.txt
>
> Not all servers use "public_html" so I can't just hard-code that as the
> web root. Is there a generic way to indicate the web folder on a server,
> the same way a tilde indicates the user's home folder? Or any other way
> to get the path?
>
> The user name and password are known.
>

Just to clariy: I tried inserting the "www" alias into the path, since
virtually all servers support that. It works with primary domains but it
fails with subdomains. Basically I'm looking for a way to force ftp to
resolve the "www.domain" part of the URL to an absolute path. My tests
so far seem to indicate that any initial "www" in the path is ignored
with ftp.

--
Jacqueline Landman Gay         |     [hidden email]
HyperActive Software           |     http://www.hyperactivesw.com

_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

Re: Converting HTTP to FTP URL

mwieder
Jacque-

How "generic" a solution are you looking for? Obviously you're not
looking for the trivial "file.txt" because if you could get it from
http why would you need ftp... Just because a site has a web server
there's no guarantee that there's also an ftp server.

So I gather that you know the username and password of a given site,
you're sure that both http and ftp services are available, but you
don't know the site address? I'm having trouble putting that together.
Can you give more details without giving away anything you don't want
to? Are you trying to upload or download (or both) files via ftp?

--
-Mark Wieder
 [hidden email]


_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
--
 Mark Wieder
 ahsoftware@gmail.com
Reply | Threaded
Open this post in threaded view
|

Re: Converting HTTP to FTP URL

Jeffrey Massung
In reply to this post by J. Landman Gay
On 08/07/2011 04:12 PM, J. Landman Gay wrote:

> I need a way to convert a web URL to a valid FTP URL. For example,
> convert this:
>
>   http://www.domain.com/folder/file.txt
>
> to this:
>
>   ftp://user:[hidden email]/webroot/folder/file.txt
>
> Not all servers use "public_html" so I can't just hard-code that as
> the web root. Is there a generic way to indicate the web folder on a
> server, the same way a tilde indicates the user's home folder? Or any
> other way to get the path?

Jacqueline,

This cannot be done unless you control the web server. A *lot* of web
servers now-a-days are not Apache (and its derivatives), but rather
routing servers like Django, Happstack, Snap, AllegroServe, etc. Routing
web frameworks allow you to literally host source files anywhere. For
example, using Happstack I can say something like (forgive the Haskell
code):

route =
   msum [ file "file.txt" $ serveFile "/home/someUsername/files/blah.wav" ]

As you can see, what was requested has absolutely nothing to do with
what was actually served to the browser.

So, the only way you can begin to solve this issue is if you have direct
access to the web server and know how it's routing and serving static
content.

Jeff M.

_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

Re: Converting HTTP to FTP URL

J. Landman Gay
On 8/7/11 10:05 PM, Jeffrey Massung wrote:

> So, the only way you can begin to solve this issue is if you have direct
> access to the web server and know how it's routing and serving static
> content.

I'll answer you and Mark Weider at the same time. I'm trying to add
auto-upload to AirLaunch. It mostly works. I can only upload via ftp
(http isn't usually accepted, I don't think, even if you provide a name
and password. At least, it doesn't work on my hosted account.) There is
already a place in the stack for the user to enter the http URL where
their installation files will be downloaded from. That URL is required
for setting up the wi-fi installation, so AirLaunch already has that.

I could ask the user for a separate FTP upload URL but I'd rather not if
I don't have to. What would be better is to take their HTTP URL and
convert it to a valid FTP URL which will only be used internally by
AirLaunch to get their files onto their server.

On both my main hosted server and at my site on on-rev, changing the
htpp URL works if I add "www" after the domain part of the path. That
allows an ftp upload to resolve the www alias, and the rest of the path
is the same as the http URL. Easy, and no hassle for the user.

But when testing with a subdomain, that method creates a new "www"
folder inside the subdomain, which isn't what's wanted. So I'd like a
generic way to create a valid FTP path from a known HTTP path, one that
works with primary domains as well as subdomains.

It sounds like I may have to ask the user to provide both paths. But
that makes everything more complex because the two paths are slightly
different, the user needs to know the distinction between web-relative
paths and absolute paths and what their web root folder is called, and
I'd rather avoid the support issues around that if I can.

--
Jacqueline Landman Gay         |     [hidden email]
HyperActive Software           |     http://www.hyperactivesw.com

_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

Re: Converting HTTP to FTP URL

Jim Ault


--- On Sun, 8/7/11, J. Landman Gay <[hidden email]> wrote:

From: J. Landman Gay <[hidden email]>
Subject: Re: Converting HTTP to FTP URL
To: "How to use LiveCode" <[hidden email]>
Date: Sunday, August 7, 2011, 8:49 PM

On 8/7/11 10:05 PM, Jeffrey Massung wrote:

> So, the only way you can begin to solve this issue is if you have direct
> access to the web server and know how it's routing and serving static
> content.

I'll answer you and Mark Weider at the same time. I'm trying to add auto-upload to AirLaunch. It mostly works. I can only upload via ftp (http isn't usually accepted, I don't think, even if you provide a name and password. At least, it doesn't work on my hosted account.) There is already a place in the stack for the user to enter the http URL where their installation files will be downloaded from. That URL is required for setting up the wi-fi installation, so AirLaunch already has that.

I could ask the user for a separate FTP upload URL but I'd rather not if I don't have to. What would be better is to take their HTTP URL and convert it to a valid FTP URL which will only be used internally by AirLaunch to get their files onto their server.

On both my main hosted server and at my site on on-rev, changing the htpp URL works if I add "www" after the domain part of the path. That allows an ftp upload to resolve the www alias, and the rest of the path is the same as the http URL. Easy, and no hassle for the user.

But when testing with a subdomain, that method creates a new "www" folder inside the subdomain, which isn't what's wanted. So I'd like a generic way to create a valid FTP path from a known HTTP path, one that works with primary domains as well as subdomains.

It sounds like I may have to ask the user to provide both paths. But that makes everything more complex because the two paths are slightly different, the user needs to know the distinction between web-relative paths and absolute paths and what their web root folder is called, and I'd rather avoid the support issues around that if I can.


-----------------------------------------------------------------------
........and now, Jaque, we are back to the begining of the circle of confusion

If subdomains are allowed on a server, then each should logically have every nested folder with separate permissions, such as
 cgi-bin/    www/   public_ftp/     private_ftp/   public_html/
(on my server, the cgi-bin/ folder is created when I create the subdomain, but it is the only one)

One question is "If a subdomain is specified, why would you want to give access to the root domain folders (that should have different permissions)?

You can have a blog root at  www.GreatLcAdventures.com/blog/
and at
Minnisota.GreatLcAdventures.com/blog/

Although the subdomain folder appears in the directory tree for the root administrator as just a folder, the server sees it as a special permissions starting point.  This means that a user that has access to only the subdomain will be restricted to that portion of the tree (of course, special tricks can change that)

As admin, if you give a user a login and passwd for the subdomain, they should only have ftp access to that folder tree.

If they have the root admin login and passwd, then they only have to keep the path names orgainized correctly.

Most of the time, ftp access for users of a subdomain has to be enabled by the admin, which will include login/passwd setup.  Assuming you only want to use the admin root access, this won't be necessary.

One answer to your quest may be that
if a subdomain is detected,
either you move that name to the folder path just after the domain.com
     
1)   www.domain.com/subName/ftpFolder/

or disregard this if you are uploading to the root ftp folder (and allowing access by all subdomains, even though the folder is outside the normal subdomain tree)
        2)   www.domain.com/ftpFolder/

My apache servers are using    /public_html/   and    ftp.domain.com/
so rather than let the server resolve the path,
I will define my explicit path as (in my case)

domains/specificDomainNameA.com/public_html/Home.php

domains/specificDomainB.com/public_html/Welcome.php


domains/specificDomainB.com/subName/Welcome.php
  (note that my server does not auto-create 'www/' for subdomains)

When you do find the answer that works for your server, document this with several examples of working and non-working strings so you can figure it out next time.  My trick is to keep all the strings that don't work, label them as such, put them into one block.  It is very easy to test the same string multiple times and lose track of the subtle differences.

I hope this does not confuse you.  The rules of the road for different servers can be tricky.

Jim Ault
Las Vegas
_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

Re: Converting HTTP to FTP URL

mwieder
In reply to this post by J. Landman Gay
Jacque-

Sunday, August 7, 2011, 8:49:52 PM, you wrote:

> It sounds like I may have to ask the user to provide both paths. But
> that makes everything more complex because the two paths are slightly
> different, the user needs to know the distinction between web-relative
> paths and absolute paths and what their web root folder is called, and
> I'd rather avoid the support issues around that if I can.

So here's my confusion about this: just because a user has http
hosting doesn't necessarily mean they have ftp hosting or that they
have an ftp account set up. And if they *do* have an ftp account and
already know their username and password, wouldn't they also know the
ftp address? And if they know all that (still with me?) why not just
ask for four pieces of data instead of three? You might try creating a
socket connection to port 21 at the server address and see if there's
a response, but that still wouldn't necessarily resolve to a subdomain
ftp server. Or you might get an ftp server but not the one you were
looking for. I think the support issues of trying to outsmart the user
might outweigh those of users asking "what's ftp?".

--
-Mark Wieder
 [hidden email]


_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
--
 Mark Wieder
 ahsoftware@gmail.com
Reply | Threaded
Open this post in threaded view
|

Re: Converting HTTP to FTP URL

Jeffrey Massung
In reply to this post by J. Landman Gay
On 08/07/2011 09:49 PM, J. Landman Gay wrote:

> On 8/7/11 10:05 PM, Jeffrey Massung wrote:
>
>> So, the only way you can begin to solve this issue is if you have direct
>> access to the web server and know how it's routing and serving static
>> content.
>
> I'll answer you and Mark Weider at the same time. I'm trying to add
> auto-upload to AirLaunch. It mostly works. I can only upload via ftp
> (http isn't usually accepted, I don't think, even if you provide a
> name and password. At least, it doesn't work on my hosted account.)
> There is already a place in the stack for the user to enter the http
> URL where their installation files will be downloaded from. That URL
> is required for setting up the wi-fi installation, so AirLaunch
> already has that.
>
> I could ask the user for a separate FTP upload URL but I'd rather not
> if I don't have to. What would be better is to take their HTTP URL and
> convert it to a valid FTP URL which will only be used internally by
> AirLaunch to get their files onto their server.

In my experience, trying to infer the location is going to lead - down
the road - to an unhappy customer. No one will remember the inference
and it will break. I can completely sympathise with what you want to do
and why. But I would advise against it.

> It sounds like I may have to ask the user to provide both paths. But
> that makes everything more complex because the two paths are slightly
> different, the user needs to know the distinction between web-relative
> paths and absolute paths and what their web root folder is called, and
> I'd rather avoid the support issues around that if I can.

If you wouldn't mind another option: use the PUT protocol in HTTP along
with authentication.
http://www.w3.org/QA/2008/10/understanding-http-put.html. This will
still require work from the customer to enable this, but it'll make life
easier for everyone.

The HTTP PUT has an added benefit: it let's the server do whatever it
wants with the incoming data. This is important, because it means that
the server can put the file into a different location and act on it
later. Consider the following setup:

1. The actual site is stored in a git repository somewhere.

2. The /var/www folder is actually a clone of the git repo.

3. For every "upload" being done (likely per user), they can be staged
somewhere else (/var/www-staged-username), which is another clone of the
repo.

4. When all the uploads are completed, a push can be made which uploads
the staged repo to the main repository.

5. A build process takes over and ensures data integrity and that
everything is correct and the upload won't bring down the live site.

6. Later on the /var/www site pulls down all changes from the repo.

This entire process can be fully automated and done behind the scenes
without any intervention or pathing "voodoo." It can also be done on a
single machine or across many machines ensuring that if one goes down or
the drive fails, the others still have all the data. And with a SCM
system in place, you'll also have full history of who did what and when.
Very handy at times. ;-)

That might be very heavy-weight for your customer [now], but it could be
a direction down the road they'd like to take things. Flesh out the
above as a plan and let them know why you'd like to support HTTP PUT and
how it would work. Perhaps now all it does it upload directly to the
live site, but explain how it sets them up for later.

Jeff M.


_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

Re: Converting HTTP to FTP URL

mwieder
Jeff-

HTTP PUT is an interesting alternative, but I see two possible
complications. First of all you'd have to query the Allow headers on
the server to see if the PUT command is supported, and then come up
with an alternate strategy if it isn't. Secondly, you'd have to deal
with the 301 response if you're being redirected to a different URI,
since the PUT command is very strict about this.

--
-Mark Wieder
 [hidden email]


_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
--
 Mark Wieder
 ahsoftware@gmail.com
Reply | Threaded
Open this post in threaded view
|

Re: Converting HTTP to FTP URL

Jeffrey Massung
On 08/08/2011 12:55 AM, Mark Wieder wrote:
> Jeff-
>
> HTTP PUT is an interesting alternative, but I see two possible
> complications. First of all you'd have to query the Allow headers on
> the server to see if the PUT command is supported, and then come up
> with an alternate strategy if it isn't. Secondly, you'd have to deal
> with the 301 response if you're being redirected to a different URI,
> since the PUT command is very strict about this.

Agreed.

FTP could work as well, but I would advise that the FTP upload be
directed somewhere known so that the rest of the "chain" stays intact.
Just use git or some other SCM so the uploads can be pushed, etc. I'd
still prefer the PUT solution over FTP (personally) because it keeps the
control of what happens in the hands of the server/customer.

Jeff M.

_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

Re: Converting HTTP to FTP URL

J. Landman Gay
In reply to this post by Jim Ault
On 8/8/11 12:32 AM, Jim Ault wrote:

> As admin, if you give a user a login and passwd for the subdomain,
> they should only have ftp access to that folder tree.

Yes, I think that would work. The problem is that the typical user will
own the whole thing, including subdomains, and the issue happens if they
log in as a main admin and try to ftp to a subdomain folder.


> When you do find the answer that works for your server, document this
> with several examples of working and non-working strings so you can
> figure it out next time.

If that was all I needed to do, it would have been finished last Friday.
:) I know how my own server is set up, but I can't predict how anyone
else's will be arranged, or which domain/subdomain they will log into.

 From the other responses, I think I need to just ask for separate paths.

--
Jacqueline Landman Gay         |     [hidden email]
HyperActive Software           |     http://www.hyperactivesw.com

_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

Re: Converting HTTP to FTP URL

J. Landman Gay
In reply to this post by mwieder
On 8/8/11 12:44 AM, Mark Wieder wrote:

> So here's my confusion about this: just because a user has http
> hosting doesn't necessarily mean they have ftp hosting or that they
> have an ftp account set up. And if they *do* have an ftp account and
> already know their username and password, wouldn't they also know the
> ftp address? And if they know all that (still with me?) why not just
> ask for four pieces of data instead of three?

Agree. And, having looked at the responses here, I agree with Jeff too
that trying to figure out a valid FTP path will cause more support
headaches than just requiring that the user figure it out.

I don't think I need four pieces of data, only two valid paths. No one
seems to have any trouble with the HTTP path, so that leaves just one
potential headache.

Not counting the one I developed myself over the weekend, of course...

--
Jacqueline Landman Gay         |     [hidden email]
HyperActive Software           |     http://www.hyperactivesw.com

_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

Re: Converting HTTP to FTP URL

mwieder
Jacque-

Monday, August 8, 2011, 8:28:27 AM, you wrote:

> I don't think I need four pieces of data, only two valid paths. No one

<g> All right. I was counting username and password as two.

--
-Mark Wieder
 [hidden email]


_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
--
 Mark Wieder
 ahsoftware@gmail.com
Reply | Threaded
Open this post in threaded view
|

Re: Converting HTTP to FTP URL

J. Landman Gay
On 8/8/11 10:44 AM, Mark Wieder wrote:
> Jacque-
>
> Monday, August 8, 2011, 8:28:27 AM, you wrote:
>
>> I don't think I need four pieces of data, only two valid paths. No one
>
> <g>  All right. I was counting username and password as two.
>

Obviously my brain has suffered after googling all weekend, and I've
never been all that good at math to start with.

--
Jacqueline Landman Gay         |     [hidden email]
HyperActive Software           |     http://www.hyperactivesw.com

_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

Re: Converting HTTP to FTP URL

Chipp Walters
Jacque,

I believe you should have a separate http path and a separate ftp path. This
is because as one adds users to their servers, they typically also add a
different root folder for the user.

So, for instance if I had a file located at:
http://fred.com/coolstack/test.rev
the ftp path for it might be: ftp://fred.com/test.rev as the username in the
ftp would direct the server to the 'coolstack' directory automatically. This
is how I solve the problem with MagicCarpet.
_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

Re: Converting HTTP to FTP URL

Stephen Barncard-4
This is true for FTP apps like Transmit , Interarchy and Cyberduck
 that all have separate fields in their settings for web URL and FTP
address. That makes features like 'copy URL' and 'copy PATH' available as a
context menu item when selecting a file in their browsers.

On 8 August 2011 13:51, Chipp Walters <[hidden email]> wrote:

> Jacque,
>
> I believe you should have a separate http path and a separate ftp path.
> This
> is because as one adds users to their servers, they typically also add a
> different root folder for the user.
>
> So, for instance if I had a file located at:
> http://fred.com/coolstack/test.rev
> the ftp path for it might be: ftp://fred.com/test.rev as the username in
> the
> ftp would direct the server to the 'coolstack' directory automatically.
> This
> is how I solve the problem with MagicCarpet.
> _______________________________________________
> use-livecode mailing list
> [hidden email]
> Please visit this url to subscribe, unsubscribe and manage your
> subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode
>



--



Stephen Barncard
San Francisco Ca. USA

more about sqb  <http://www.google.com/profiles/sbarncar>
_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode