Projects tagged ‘curl’ and ‘http’

curl is a command line tool for transferring files with URL syntax, supporting FTP, FTPS, TFTP, HTTP, HTTPS, SCP, SFTP, TELNET, DICT, FILE and LDAP. curl supports SSL certificates, HTTP POST, HTTP ... [More] PUT, FTP uploading, HTTP form based upload, proxies, cookies, user+password authentication (Basic, Digest, NTLM, Negotiate, kerberos...), file transfer resume, proxy tunneling and a busload of other useful tricks. [Less]

PycURL is a Python interface to libcurl. PycURL can be used to fetch objects identified by a URL from a Python program, similar to the urllib Python module. PycURL is mature, very fast, and supports a lot of features.

Multi-connection command line tool to download Internet sites. Similar to wget and cURL, but it manages up to 50 parallel links. Main features are: recursive fetching, Metalink retrieving, segmented download and image filtering by width and height.

Write and run functional tests of your web application using resources available on almost every Linux host. bashWebTest is an esoteric framework that started as part dare, part experiment. At this ... [More] point the tool has become somewhat useful for my situation so I figure it's a good time to let others check it out. I am giving back to the internets. The crudest, ugliest code in the world is available right now. Take a peek if only you're terribly desperate for a lame testing tool. [Less]

A more efficient implementation of curl_multi() curl_multi is a great way to process multiple HTTP requests in parallel in PHP. curl_multi is particularly handy when working with large data sets (like ... [More] fetching thousands of RSS feeds at one time). Unfortunately there is very little documentation on the best way to implement curl_multi. As a result, most of the examples around the web are either inefficient or fail entirely when asked to handle more than a few hundred requests. The problem is that most implementations of curl_multi wait for each set of requests to complete before processing them. If there are too many requests to process at once, they usually get broken into groups that are then processed one at a time. The problem with this is that each group has to wait for the slowest request to download. In a group of 100 requests, all it takes is one slow one to delay the processing of 99 others. The larger the number of requests you are dealing with, the more noticeable this latency becomes. The solution is to process each request as soon as it completes. This eliminates the wasted CPU cycles from busy waiting. I also created a queue of cURL requests to allow for maximum throughput. Each time a request is completed, I add a new one from the queue. By dynamically adding and removing links, we keep a constant number of links downloading at all times. This gives us a way to throttle the amount of simultaneous requests we are sending. The result is a faster and more efficient way of processing large quantities of cURL requests in parallel. [Less]

What for?To be able to get web pages content with PHP easily FeaturesVery simple HTTP Headers/cookies/body retrieving Session browsing Regular expression matches (PCRE syntax) Different encodings ... [More] support Binary data retrieving RequirementsPHP 5.x cURL PHP extension Computer Quick startDownload latest stable version Unpack (tar -xvf phrowser.tar.gz) Run examples from CLI: cd examples php example.google.phpWrite your own script: url_get('http://google.com/'); # Match google's page title echo $b->find_scalar_match('/(.*?)<\/title>/i');You should see "Google" after running that script DocumentationApiManual [Less]

RequestCore is a lightweight cURL-based HTTP request/response class that leverages MultiCurl for parallel requests. RequestCore was born out of the Tarzan project, and was designed from the start ... [More] to be a reliable standalone component to be re-used in any other project. [Less]

CurlKit - Objective C interface for remote file systemsThis project compiles a number of client networking libraries into a re-usable framework called CurlKit for downloading and uploading files from ... [More] remote servers using protocols such as HTTP, FTP and SFTP. The framework can be used directly in your own applications. It's currently in development, and mainly includes support for SFTP connections at the moment. There's more information in the Introduction page. There is a much better project http://code.google.com/p/curlhandle/ which is a fuller implementaton of an Objective-C interface to libcurl. I was going to make this an Objective-C libcurl framework, but decided for the meantime to focus my efforts on basic file transfer protocols first. [Less]

A fork of the s3-bash project, adding features like creating and removing buckets, and improving the interface a little. The original s3-bash code is licensed under the Apache License 2.0. This ... [More] code simply adds a better interface with more functions on top of it, and that interface is licensed under MIT license. [Less]