mircscripting.info Forum Index mircscripting.info
#mIRCscripting Forum
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

Sockets and HTTP servers...

 
Post new topic   Reply to topic    mircscripting.info Forum Index -> Tutorials
View previous topic :: View next topic  
Author Message
Violent-J
earlybird


Joined: 16 Jan 2004
Posts: 2
Location: Texas

PostPosted: Fri Jan 16, 2004 9:17 pm    Post subject: Sockets and HTTP servers... Reply with quote

One of my serious lacks of knowledge comes when I have to deal with sockets, but after hours of trudging through the mess of them I've come to realize why they are sometimes so damned hard.

I had aspirations of just going into the mIRC help file and learning them in a snap, boy was I in over my head. There is one thing you need to realize with sockets... not only are you learning how to open/close/read/write to them, you must also learn the protocol of whatever it is that you are trying to access.

In this tutorial we're going to look at accessing an HTTP server, both with no access limited settings and with. To start of with we use the beloved default line:

Code:

on 1:sockread:testing:{
  if ($sockerr > 0) return
  :nextread
  sockread %temp
  if ($sockbr == 0) return
  if (%temp == $null) %temp = -
  echo 4 %temp
  goto nextread
}


Memorize this line, then... memorize again... you'll always need this, or at least something very similar. Now about what this is exactly... anytime that you open a socket and write soemthing to it, whatever you're communicating with is going to send some kind of a response back. This line will read what it sends and display it to you.

Now here comes the nitty gritty stuff. After researching, screaming, kicking, clawing, eating, drinking, sleeping, & more researching I finally came across the information I needed to retrieve a file from a webserver. Here is the lines that I used (I'll go over each individual part below)

Code:

on 1:sockopen:testing: {
  sockwrite -n testing GET /index.html HTTP/1.0
  sockwrite -n testing Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, */*
  sockwrite -n testing User-Agent: Mozilla/4.0 (compatible; MSIE 5.5; Windows 98; DigExt)
  sockwrite -n testing Connection: Keep-Alive
  sockwrite -n testing Host: www.website.com
  sockwrite -n testing Authorization: Basic ZmFrZTpwYXNzd29yZA==
  sockwrite testing $crlf
}


Let's start at the begining:

on 1:sockopen:testing: {
-This line basically says "When I open a socket called "testing" here's what I want you to do."

sockwrite -n testing GET /index.html HTTP/1.0
-This line tells the webserver to send the file "index.html" using the HTTP/1.0 porotocol

sockwrite -n testing Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, */*
-This line basically says to the web server that it's capable of accepting anything and everything

sockwrite -n testing User-Agent: Mozilla/4.0 (compatible; MSIE 5.5; Windows 98; DigExt)
-This is probably the most important line that you will be sending to the webserver. As I came to find out a webserver is looking for mostly 1 important thing in this line... "Mozilla/4.0" This tells the server what kind of client you are using and as such to send data accordingly.

sockwrite -n testing Connection: Keep-Alive
-Standard keep-alive request

sockwrite -n testing Host: www.website.com
-This line is important to change to the address of the website that you are accessing, so say we're trying to get mircscripting's website... we'd change www.website.com to mircscripting.info

Optional Step:
sockwrite -n testing Authorization: Basic YWRtaW46cGxiZ2E1NDM=
-This is a very important part if and only if you need to be authenticated to view a website, the jargon you see after the word "Basic" is actually an encrypted password, in order to get this you need to have either the .htpasswd file from the server or use the generator that comes with the Windows version of Apache (pm/e-mail me if u need further help with this)

Now that is what you need in order to access the file: index.html from a webserver. Remember, if you want to access something else, just change the file name.

Now as for what you can now do with this info, it's entirely up to you. I used this partiuclar one to grab an XML document from my webserver that contained information about my ShoutCAST stream.

That's about it for my first tutorial, I hopy you find it helpful.

P.S. Quick reminder on how to actually open the socket: /sockopen <sockname> <site> <port> e.x. /sockopen testing www.website.com 80
_________________
Violent-J


Last edited by Violent-J on Thu May 26, 2005 11:44 am; edited 2 times in total
Back to top
View user's profile Send private message
holospoof
Major


Joined: 14 Jan 2004
Posts: 103

PostPosted: Sat Jan 24, 2004 8:44 am    Post subject: Reply with quote

very nice Tutorial! I did it all the hard way (looking at RFC's) but only because I came across this later! Mad

TWO THUMBS UP!
_________________
blah!^!@^$^^
Back to top
View user's profile Send private message Visit poster's website
unreal
earlybird


Joined: 13 Feb 2004
Posts: 6
Location: England

PostPosted: Sat Feb 14, 2004 11:18 am    Post subject: Reply with quote

Quote:
sockwrite -n testing User-Agent: Mozilla/4.0 (compatible; MSIE 5.5; Windows 98; DigExt)
-This is probably the most important line that you will be sending to the webserver. As I came to find out a webserver is looking for mostly 1 important thing in this line... "Mozilla/4.0" This tells the server what kind of client you are using and as such to send data accordingly.

The HTTP is a standard, so unless the HTTP daemon was written by an idiot, it should send the same data to the client whether or not they're using IE, Lynx, Opera or some other client. The User-Agent header isn't required, it's just an extra chunk of data to be put into the HTTPd logs on each request, to know what sort of clients users are accessing the file with.

But nonetheless, nice tutorial Smile
_________________
Fly like a mouse. Run like a cushion. Be the small bookcase.


Last edited by unreal on Sun Feb 15, 2004 6:26 pm; edited 1 time in total
Back to top
View user's profile Send private message
Petah
Major


Joined: 27 Dec 2003
Posts: 59
Location: New Zealand

PostPosted: Sun Feb 15, 2004 9:41 am    Post subject: Reply with quote

I find the best way to work out how to use sockets is if you know the basics you can make a small scipt like this
Code:
alias gogogo {
  socklisten IE 80
  sockopen google google.com 80
}
on *:sockopen:google: {
  .timer 1 1 url http:// $+ $ip
}
on *:sockread:google: {
  sockread %in | tokenize 32 %in
  sockwrite -n IE $1-
  echo -a GOOGLE: $1-
}
on *:sockread:IE: {
  sockread %in | tokenize 32 %in
  echo -a IE: $1-
  sockwrite -n google $1-
}

THIS ISNT A WORKING SCRIPT
it was just to give you a idea of what i mean
Back to top
View user's profile Send private message Send e-mail MSN Messenger
unreal
earlybird


Joined: 13 Feb 2004
Posts: 6
Location: England

PostPosted: Mon Feb 16, 2004 11:34 am    Post subject: Reply with quote

If you don't mind me asking, if it isn't a working script, why did you post it when it could give beginners the wrong idea?
_________________
Fly like a mouse. Run like a cushion. Be the small bookcase.
Back to top
View user's profile Send private message
moneo
Major


Joined: 27 Dec 2003
Posts: 58
Location: Savannah, GA, USA

PostPosted: Mon Feb 16, 2004 1:35 pm    Post subject: Reply with quote

Even if written correctly, I doubt a beginner would know how to use it.

Petah, that's a very good idea. I've thought about writing something similar, but never got around to it. It's a great way to play "data tag"--getting data, checking what the proper response is, and then sending that response back. Basically, it makes a complete log of data transfer between the browser and the site, similar to mIRC's /debug.

Maybe you should turn this into an addon with some options. :-D
_________________

me@lucasoman.com | http://lucasoman.com
Back to top
View user's profile Send private message Send e-mail Visit poster's website AIM Address Yahoo Messenger MSN Messenger
ralarX
Quite Active


Joined: 23 Apr 2005
Posts: 45

PostPosted: Sat Apr 23, 2005 4:35 am    Post subject: Reply with quote

Some weeks ago i tried to make a kinda 'universal' http reader, it should be able to read every page, regardless layout:

Code:

alias gethttp {
  if (!$1) { return }
  if ($2 isnum) { var %port = $2 } | else { var %port = 80 }
  if (http://* iswm $1) { var %url = $gettok($1,2-,47) }
  else { var %url = $1 }
  var %id = $unisock_rs(8) | var %sockname = unisocket. $+ %id | sockopen %sockname $gettok(%url,1,47) %port | sockmark %sockname %url
}
on *:SOCKOPEN:unisocket.*:{
  if ($sockerr) { echo -s Socket $sockname $+ : Open - $sock($sockname).wsmsg $+ . $sock($sockname).mark | return }
  var %url = $sock($sockname).mark, %w = @ $+ $gettok($sockname,2,46) | if ($window(%w)) { clear %w } | else { window -de %w }
  ; sockwrite -n $sockname Content-Length: $calc($len(%msg2) + $len(%msg3))
  sockwrite -n $sockname GET / $+ $gettok(%url,2-,47) HTTP/1.1
  sockwrite -n $sockname Host: $gettok(%url,1,47)
  sockwrite -n $sockname user-agent: Mozilla/??
  sockwrite -n $sockname Connection: Keep-Alive
  sockwrite -n $sockname $crlf
}

on *:SOCKCLOSE:unisocket.*: {
  if ($hget(HSockBuffer)) { hdel HSockBuffer $sockname }
  echo -s Socket $sockname $+ : Connection closed remotely. $sock($sockname).mark
}
on *:SOCKREAD:unisocket.*: {
  if ($sockerr > 0) { var %m = Read - $sock($sockname).wsmsg $+ . | goto closesocket }
  ; Next 2 variables define the <start of line> <end of line> strings.
  ; If only one (an ending carriage return line feed $crlf is commonly used) then use same values for both.
  ; If no tags (just display entire content) use 'usnone' without the quotes for both tags.
  var %tag1 = usnone
  var %tag2 = usnone

  if (%tag1 == usnone) && (%tag2 == usnone) { var %chunksize = $sock($sockname).rq | sockread -f %chunksize &chunk | if ($sockbr) { echo -s OK | goto output } | return }
  var %tag1len = $len(%tag1), %tag2len = $len(%tag2) | if (!$hget(HSockBuffer)) { hmake HSockBuffer 30 }
  .echo -q $hget(HSockBuffer,$sockname,&buffer) | sockread -f $sock($sockname).rq &chunk | if ($sockbr == 0) { return }
  bcopy &buffer $calc($bvar(&buffer,0) + 1) &chunk 1 -1 | bunset &chunk | var %bufsize = $bvar(&buffer,0), %offset = 1

  :nextframe
  var %postag1 = $bfind(&buffer,%offset,%tag1).text | if (!%postag1) { goto noframe }
  if (%tag1 == %tag2) {
    var %chunkstart = $calc(%postag1 + %tag1len), %postag2 = $bfind(&buffer,%chunkstart,%tag2).text | if (!%postag2) { goto noframe }
    goto setframe
  }
  var %postag2 = $bfind(&buffer,%offset,%tag2).text | if (!%postag2) { goto noframe }
  if (%postag2 < %postag1) { var %offset = $calc(%postag2 + %tag2len) | goto nextframe }
  var %chunkstart = $calc(%postag1 + %tag1len) | goto setframe

  :noframe
  if (%bufsize > 16384) { var %m = too many bytes until ordered tag couple (tag1: %tag1 tag2: %tag2 $+ ). | goto closesocket }
  elseif (%offset < %bufsize) { bcopy -c &buffer 1 &buffer %offset $calc(%bufsize - %offset + 1) | hadd -b HSockBuffer $sockname &buffer }
  else { hdel HSockBuffer $sockname }
  bunset &buffer | return

  :setframe

  var %chunkend = $calc(%postag2 - 1), %chunksize = $calc(%chunkend - %chunkstart + 1) | bcopy -c &chunk 1 &buffer %chunkstart %chunksize

  :output

  ; processing example: append to file
  ; bwrite somefile.ext -1 -1 &chunk

  ; processing example: append to hash UniSocket
  if (!$hget(UniSocket)) { hmake UniSocket 30 }
  .echo -q $hget(UniSocket,$sockname,&unitemp) | bcopy &unitemp $calc($bvar(&unitemp,0) + 1) &chunk 1 -1
  hadd -b UniSocket $sockname &unitemp | bunset &unitemp

  ; processing example: display in lines of 800 to custom window
  var %i = 1, %lsize = 800, %bytestodo = 0, %w = @ $+ $gettok($sockname,2,46) | window -de %w | titlebar %w $sock($sockname).mark
  while (%i <= %chunksize) { aline -p %w $bvar(&chunk,%i,%lsize).text | inc %i %lsize }
  if (%chunksize > %lsize) {
    var %lines = $int($calc(%chunksize / %lsize)), %bytesdone = $calc(%lines * %lsize), %bytestodo = $calc(%chunksize - %bytesdone)
    if (%bytestodo > 0) && ($bvar(&chunk,$calc(%bytesdone + 1),%bytestodo).text != $null) { aline -p %w $v1 }
  }

  if (%tag1 == usnone) && (%tag2 == usnone) { return }
  var %offset = $calc(%postag2 + %tag2len) | goto nextframe
  :closesocket
  echo -s Socket $sockname $+ : %m Cleaning up & closing socket> $sock($sockname).mark
  hdel HSockBuffer $sockname | sockclose $sockname
}
alias -l unisock_rs { return $eval($str($ $+ rand(a,z) $ $+ + $ $+ + $chr(32),$1),2) }
alias unisock_clear { if ($hget(UniSocket)) { hfree UniSocket } }
alias unisock_search {
  var %total = $hget(UniSocket,0).item, %i = 1, %count = 0
  while (%i <= %total) {
    var %item = $hget(UniSocket,%i).item | .echo -q $hget(UniSocket,%item,&unitemp)
    if ($bfind(&unitemp,1,$1).text != $null) { inc %count }
    bunset &unitemp | inc %i
  }
  return %count
}


Usage:
-> Start retrieval:
/gethttp url [optional port]
if port not specified, defaults to 80.
Example:
//gethttp http://web-sniffer.net/?url=http%3A%2F%2Fimages.google.dk%2Fimages%3Fq%3Dhest%26hl%3Dda%26btnG%3DS%25C3%25B8g%2Bi%2Bbilleder&submit=Submit&http=1.1&gzip=yes&type=GET&ua=Mozilla%2F5.0+%28Windows%3B+U%3B+Windows+NT+5.1%3B+da%3B+rv%3A1.8.0.4%29+Gecko%2F20060508+Firefox%2F1.5.0.4+Web-Sniffer%2F1.0.24

-> Check if certain text is present in any of the retrieved webpages buffer:
/unisock_search <text>

-> Clear retrieved webpages buffer:
/unisock_clear


Last edited by ralarX on Sat Jun 10, 2006 3:52 pm; edited 9 times in total
Back to top
View user's profile Send private message
Violent-J
earlybird


Joined: 16 Jan 2004
Posts: 2
Location: Texas

PostPosted: Thu May 26, 2005 11:41 am    Post subject: Reply with quote

For those of you a bit confused about the Authorization header aspect of this code snippet here you go...

Quote:

The HTTP/1.0 specification defines the BASIC authorization scheme, where the authorization parameter is the string of username:password encoded in base 64. For example, for the username of "webmaster" and a password of "zrma4v," the authorization header would look like this:

Authorization: BASIC d2VibWFzdGVyOnpycW1hNHY=

The value decodes into webmaster:zrma4v.


Source: http://www.oreilly.com/openbook/webclient/appa.html
_________________
Violent-J
Back to top
View user's profile Send private message
IL_Pizu
Major


Joined: 28 Dec 2003
Posts: 113
Location: .:Europe:. .:Malta:.

PostPosted: Mon Jun 13, 2005 1:46 am    Post subject: Reply with quote

cool tutorial... i like Wink
_________________
Check This Out: http://www.shaddowdomain.com/scary.jpg Smile
http://pizu.modblog.com/ Smile
Back to top
View user's profile Send private message Visit poster's website MSN Messenger
DeMeteor
Guest





PostPosted: Wed Aug 22, 2007 2:53 am    Post subject: Reply with quote

Thanks thats was pretty helpfull
i was trying to learn then some days ago for one remote of mine :p
thanks
Back to top
Display posts from previous:   
Post new topic   Reply to topic    mircscripting.info Forum Index -> Tutorials All times are GMT - 5 Hours
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2005 phpBB Group