C Programming Language :: Accessing web with a C program


	Accessing web with a C program Hi, I want to write a program that essentially connects to web, access some web page and pass some parameters and fetch the return values and print like LWP in Perl Context. Can some one suggest me what are the headers I might require and directions to such a documentation. Thanks, Vasundhar Neo wrote: > Hi, > I want to write a program that essentially connects to web, > access some web page and pass some parameters and fetch the return > values and print > like LWP in Perl Context. > Can some one suggest me what are the headers I might require and > directions to such a documentation. The functionality you require isn't part of standard C, but is part of your operating environment. The best place to ask this question is on a group dedicated to your platform of choice. -- Ian Collins. On Feb 8, 3:22 pm, "Neo" <bvasund@gmail.com> wrote: > Hi, > I want to write a program that essentially connects to web, > access some web page and pass some parameters and fetch the return > values and print > like LWP in Perl Context. > Can some one suggest me what are the headers I might require and > directions to such a documentation. Given the proper compliant implementation, you should only need #include <stdio.h> You /should/ be able to FILE web = fopen("http://google.com/","r"); and have a valid read connection to the named web page. However, few (if any) compliant C implementations provide this sort of interpretation of the filename string. If you find that yours does not, then you'll have to look for a compiler that does. Otherwise, you are stuck with non-standard (as far as CLC goes) libraries that you will have to discuss in the appropriate newsgroup (but not here). HTH -- Lew Lew Pitcher <lpitc@sympatico.ca> wrote: > You /should/ be able to > FILE web = fopen("http://google.com/","r"); > and have a valid read connection to the named web page. Should? I'm curious, what argues for this behavior? I certainly wouldn't expect it from fopen() regardless of whether it was commonly implemented or not. -- C. Benson Manica \| I should know what I'm talking about - if I cbmanica(at)gmail.com \| don't, I need to know. Flames welcome. - Hide quoted text - Lew Pitcher wrote: > On Feb 8, 3:22 pm, "Neo" <bvasund@gmail.com> wrote: >> Hi, >> I want to write a program that essentially connects to web, >> access some web page and pass some parameters and fetch the return >> values and print >> like LWP in Perl Context. >> Can some one suggest me what are the headers I might require and >> directions to such a documentation. > Given the proper compliant implementation, you should only need > #include <stdio.h> > You /should/ be able to > FILE web = fopen("http://google.com/","r"); > and have a valid read connection to the named web page. Wow. stdio.h has really been gussied up since the last time I looked. Neo wrote: > I want to write a program that essentially connects to web, > access some web page and pass some parameters and fetch the return > values and print > like LWP in Perl Context. > Can some one suggest me what are the headers I might require and > directions to such a documentation. You need to look at the various HTTP protocol standards, among others. You can probably find the source for curl and wget to give you pointers. Google knows about all this. Why are you reinventing this particular wheel? On Thu, 8 Feb 2007 20:40:54 +0000 (UTC), in comp.lang.c , Christopher Benson-Manica <a@otaku.freeshell.org> wrote: >Lew Pitcher <lpitc@sympatico.ca> wrote: >> You /should/ be able to >> FILE web = fopen("http://google.com/","r"); >> and have a valid read connection to the named web page. >Should? I'm curious, what argues for this behavior? One argument would be that since quite a long time ago, filesystems have been distributed over different hardware. In that context, http:// is no less a valid identifier of some location on another machine than DRA1: or /mnt/hda1 or \\someserver or whatever weirdness one would do on MVS to reference a disk on a remote server. > I certainly >wouldn't expect it from fopen() regardless of whether it was commonly >implemented or not. Its an interesting point though. Why /should/ we expect fopen() to handle a UNC or a mount point but not a URL? -- Mark McIntyre "Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it." --Brian Kernighan Christopher Benson-Manica wrote, On 08/02/07 20:40: > Lew Pitcher <lpitc@sympatico.ca> wrote: >> You /should/ be able to >> FILE web = fopen("http://google.com/","r"); >> and have a valid read connection to the named web page. > Should? I'm curious, what argues for this behavior? The notation Lew has used is standard notation for specifying a web page and is supported by a number of tools people use for browsing files on their local file systems, therefore it would seem a logical way of adding support for network access of files to standard C programs with no need to change those programs. If could also support ftp:// smb:// etc. > I certainly > wouldn't expect it from fopen() regardless of whether it was commonly > implemented or not. I would not expect it either, but as Lew said short of the implementation providing that functionality the OP will have to go to system specific libraries. -- Flash Gordon Clever Monkey <clvrmnky.inva@hotmail.com.invalid> writes: > Lew Pitcher wrote: [...] >> You /should/ be able to >> FILE web = fopen("http://google.com/","r"); >> and have a valid read connection to the named web page. > Wow. stdio.h has really been gussied up since the last time I looked. Supporting URLs as file names shouldn't require any changes to stdio.h. It would require substantial changes to the implementation of fopen(), of course (unless the fopen() implementation in question already provides this). Whether stdio should support this is another question. -- Keith Thompson (The_Other_Keith) k@mib.org <http://www.ghoti.net/~kst> San Diego Supercomputer Center <> <http://users.sdsc.edu/~kst> We must do something. This is something. Therefore, we must do this. Mark McIntyre wrote: ... snip ... > Its an interesting point though. Why /should/ we expect fopen() > to handle a UNC or a mount point but not a URL? We don't. All we know is (from N869): 7.19.5.3 The fopen function Synopsis [#1] #include <stdio.h> FILE fopen(const char * filename, const char * mode); Description [#2] The fopen function opens the file whose name is the string pointed to by filename, and associates a stream with it. Which says nothing whatsoever about the format of 'filename'. That is system dependent. -- <http://www.cs.auckland.ac.nz/~pgut001/pubs/vista_cost.txt> <http://www.securityfocus.com/columnists/423> "A man who is right every time is not likely to do very much." -- Francis Crick, co-discover of DNA "There is nothing more amazing than stupidity in action." -- Thomas Matthews In article <lnmz3ob7g3.@nuthaus.mib.org>, Keith Thompson <k@mib.org> wrote: >>> You /should/ be able to >>> FILE web = fopen("http://google.com/","r"); >>> and have a valid read connection to the named web page. >> Wow. stdio.h has really been gussied up since the last time I looked. >Supporting URLs as file names shouldn't require any changes to >stdio.h. It would require substantial changes to the implementation >of fopen(), of course (unless the fopen() implementation in question >already provides this). A practical implementation would probably choose to add quite a lot of additional functionality. For example, it would probably want to provide a way to determine the mime type and base URI of a resource that had been opened in this way. Many protocols require authentication of some kind - in the case of writing, almost always. There are innumerable options related to caching and so on. But I think it might well be possible to do it all in such a way that a useful class of programs could indeed open files for reading just by using fopen(). -- Richard -- "Consideration shall be given to the need for as many as 32 characters in some alphabets" - X3.4, 1963. On Feb 8, 3:40 pm, Christopher Benson-Manica <a@otaku.freeshell.org> wrote: > Lew Pitcher <lpitc@sympatico.ca> wrote: > > You /should/ be able to > > FILE web = fopen("http://google.com/","r"); > > and have a valid read connection to the named web page. > Should? I'm curious, what argues for this behavior? I certainly > wouldn't expect it from fopen() regardless of whether it was commonly > implemented or not. What sort of behaviour would you expect from FILE somefile = fopen("C:\\ABC\\DEF\\GHI.J","r"); or FILE somefile = fopen("DDNAME:SYSIN,"r"); or FILE somefile = fopen("/a/b/c.d","r"); ? What's different (wrt how fopen() is defined in the standard, or how it is used in real life) betwen these usages and the usage I suggest? What is your objection to FILE web = fopen("http://google.com/","r"); and why don't you voice similar objections to FILE somefile = fopen("C:\\ABC\\DEF\\GHI.J","r"); or FILE somefile = fopen("DDNAME:SYSIN,"r"); ? On Feb 9, 1:22 am, "Neo" <bvasund@gmail.com> wrote: > Hi, > I want to write a program that essentially connects to web, > access some web page and pass some parameters and fetch the return > values and print > like LWP in Perl Context. you may want to use, 1. libwww or curl or simillar stuff.(Google curl or libwww) which is doing simillar stuff. 2. invoke external program from your c program 3. Use Browser API (if they are providing) I think option 1 is good. Ignore above option if you feel i dont understand your question properly. > Can some one suggest me what are the headers I might require and > directions to such a documentation. Third party libraries/headers > Thanks, > Vasundhar --raxit - Hide quoted text - "Lew Pitcher" <lpitc@sympatico.ca> wrote: > On Feb 8, 3:40 pm, Christopher Benson-Manica > <a@otaku.freeshell.org> wrote: > > Lew Pitcher <lpitc@sympatico.ca> wrote: > > > You /should/ be able to > > > FILE web = fopen("http://google.com/","r"); > > > and have a valid read connection to the named web page. > > Should? I'm curious, what argues for this behavior? I certainly > > wouldn't expect it from fopen() regardless of whether it was commonly > > implemented or not. > What sort of behaviour would you expect from > FILE somefile = fopen("C:\\ABC\\DEF\\GHI.J","r"); > or > FILE somefile = fopen("DDNAME:SYSIN,"r"); > or > FILE somefile = fopen("/a/b/c.d","r"); I'd expect it to open a local file. This is very different from opening a URL, because that involves making a connection to a random machine on the 'net, and possibly getting the user to enter dialing passwords and all. > What's different (wrt how fopen() is defined in the standard, or how > it is used in real life) betwen these usages and the usage I suggest? What's different between cat /home/bloggsj/agenda and cat http://www.google.com/ entered on a normal Unixoid command line? > What is your objection to > FILE web = fopen("http://google.com/","r"); I have no objection to it /per se/; but I do think that claiming that you _should_ be able to do it is short-sighted. It would be a nice enhancement in those environments where it makes sense; but it's hardly a life-saver, and there are enough snags to it that it can hardly be expected of most systems. Richard - Hide quoted text - Mark McIntyre wrote: > On Thu, 8 Feb 2007 20:40:54 +0000 (UTC), in comp.lang.c , Christopher > Benson-Manica <a@otaku.freeshell.org> wrote: > >Lew Pitcher <lpitc@sympatico.ca> wrote: > >> You /should/ be able to > >> FILE web = fopen("http://google.com/","r"); > >> and have a valid read connection to the named web page. > >Should? I'm curious, what argues for this behavior? > One argument would be that since quite a long time ago, filesystems > have been distributed over different hardware. > In that context, http:// is no less a valid identifier of some > location on another machine than DRA1: or /mnt/hda1 or \\someserver or > whatever weirdness one would do on MVS to reference a disk on a remote > server. In that case, it's an O/S issue, and not a C issue. > > I certainly > >wouldn't expect it from fopen() regardless of whether it was commonly > >implemented or not. > Its an interesting point though. Why /should/ we expect fopen() to > handle a UNC or a mount point but not a URL? Again, ask the O/S designers why they support UNC but not URL. On the platforms that I use for which I have C runtime library source, the C runtime library passes the "filename" as-is to the O/S. If a UNC filename works, it is because the O/S itself will handle the UNC parts, just as it is the O/S that handles any directory tree parts. -- +-------------------------+--------------------+-----------------------+ \| Kenneth J. Brody \| www.hvcomputer.com \| #include \| \| kenbrody/at\spamcop.net \| www.fptech.com \| <std_disclaimer.h> \| +-------------------------+--------------------+-----------------------+ Don't e-mail me at: <mailto:ThisIsASpamT@gmail.com> - Hide quoted text - Mark McIntyre wrote: > On Thu, 8 Feb 2007 20:40:54 +0000 (UTC), in comp.lang.c , Christopher > Benson-Manica <a@otaku.freeshell.org> wrote: > >Lew Pitcher <lpitc@sympatico.ca> wrote: > >> You /should/ be able to > >> FILE web = fopen("http://google.com/","r"); > >> and have a valid read connection to the named web page. > >Should? I'm curious, what argues for this behavior? > One argument would be that since quite a long time ago, filesystems > have been distributed over different hardware. > In that context, http:// is no less a valid identifier of some > location on another machine than DRA1: or /mnt/hda1 or \\someserver or > whatever weirdness one would do on MVS to reference a disk on a remote > server. > > I certainly > >wouldn't expect it from fopen() regardless of whether it was commonly > >implemented or not. > Its an interesting point though. Why /should/ we expect fopen() to > handle a UNC or a mount point but not a URL? Maybe because the origins of fopen() predate the web by a decade. The designer/s of fopen() probably couldn't forsee so that in a decade "the network would be the computer." On Thu, 08 Feb 2007 18:15:50 -0500, in comp.lang.c , CBFalconer <cbfalco@yahoo.com> wrote: >Mark McIntyre wrote: >... snip ... >> Its an interesting point though. Why /should/ we expect fopen() >> to handle a UNC or a mount point but not a URL? >We don't. All we know is (from N869): (snipped) >Which says nothing whatsoever about the format of 'filename'. That >is system dependent. Exactly my point. There's nothing in the C standard that prohibits, or encourages, fopen from handling URLs any differently from other filenames. -- Mark McIntyre "Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it." --Brian Kernighan On Fri, 09 Feb 2007 11:34:17 GMT, in comp.lang.c , r@hoekstra-uitgeverij.nl (Richard Bos) wrote: >"Lew Pitcher" <lpitc@sympatico.ca> wrote: >> What sort of behaviour would you expect from >> FILE somefile = fopen("C:\\ABC\\DEF\\GHI.J","r"); >> or >> FILE somefile = fopen("DDNAME:SYSIN,"r"); >> or >> FILE somefile = fopen("/a/b/c.d","r"); >I'd expect it to open a local file. Then you don't know as much about unix as you thought you did. On my machine, /apps/shared is a mountpoint on a netapp filer somewhere about a hundred miles from me. I can also access this from my Windows box as f:\shared. >This is very different from opening a URL, because that involves making >a connection to a random machine on the 'net, and possibly getting the >user to enter dialing passwords and all. As does the opening of a file on a networked drive, or through a mount. There is no difference. -- Mark McIntyre "Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it." --Brian Kernighan On Fri, 09 Feb 2007 13:06:45 -0500, in comp.lang.c , Kenneth Brody <kenbr@spamcop.net> wrote: >In that case, it's an O/S issue, and not a C issue. Strictly, its an implementation issue. An implementation of the C library would be allowed to do this. >> Its an interesting point though. Why /should/ we expect fopen() to >> handle a UNC or a mount point but not a URL? >Again, ask the O/S designers why they support UNC but not URL. Again, ask the implementors, too. >On the platforms that I use for which I have C runtime library source, >the C runtime library passes the "filename" as-is to the O/S. If a >UNC filename works, it is because the O/S itself will handle the UNC >parts, just as it is the O/S that handles any directory tree parts. shrug. It happens that your OSen handle UNCs. If they didn't, your library implementor could still support them. -- Mark McIntyre "Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it." --Brian Kernighan - Hide quoted text - r@hoekstra-uitgeverij.nl (Richard Bos) writes: > "Lew Pitcher" <lpitc@sympatico.ca> wrote: >> On Feb 8, 3:40 pm, Christopher Benson-Manica >> <a@otaku.freeshell.org> wrote: >> > Lew Pitcher <lpitc@sympatico.ca> wrote: >> > > You /should/ be able to >> > > FILE web = fopen("http://google.com/","r"); >> > > and have a valid read connection to the named web page. >> > Should? I'm curious, what argues for this behavior? I certainly >> > wouldn't expect it from fopen() regardless of whether it was commonly >> > implemented or not. >> What sort of behaviour would you expect from >> FILE somefile = fopen("C:\\ABC\\DEF\\GHI.J","r"); >> or >> FILE somefile = fopen("DDNAME:SYSIN,"r"); >> or >> FILE somefile = fopen("/a/b/c.d","r"); > I'd expect it to open a local file. How do you know that "/a/b/c.d" is the name of a local file? It could well be stored on some server somewhere and accessed via NFS or some other protocol. > This is very different from opening a URL, because that involves making > a connection to a random machine on the 'net, and possibly getting the > user to enter dialing passwords and all. It's not obvious that fopen() should support the NFS protocol but not the HTTP protocol. There is a difference in that an HTTP name (a URL) specifally names the remote server -- but some systems, including VMS, have file names that do the same thing but are expected to be treated as ordinary files. It's also possible, on some operating systems, to create a filesystem that maps URLs to what appear to be local file names. On such a system, fopen("http://google.com", "r") would invoke code in the filesystem driver that would access google.com using the HTTP protocol, but that would be invisible to the program that calls fopen(). Given this capability letting fopen() accept URLs directly is just a matter of doing the net access at a different level. Of course, you have to draw the line somewhere. The usual practice is not to treat URLs as file names. In my opinion, that's not a bad place to draw the line, but I don't see any strong arguments for or against drawing it somewhere else. -- Keith Thompson (The_Other_Keith) k@mib.org <http://www.ghoti.net/~kst> San Diego Supercomputer Center <*> <http://users.sdsc.edu/~kst> We must do something. This is something. Therefore, we must do this.