Created attachment 154629 [details]
Python proxy to confirm the bug or be used as a workaround

I have a freshly installed FreeBSD 10.1 at work in a VM. As it is behind a corporate proxy, I had to set HTTP_PROXY in environment and it began to run fine.

But no way to get pkg to work correctly. I installed it from ports, still same issue :

root@FriBi:~ # pkg update -f
Updating FreeBSD repository catalogue...
pkg: repository meta /var/db/pkg/FreeBSD.meta has wrong version or wrong format
Fetching meta.txz:   0%
pkg: No signature found
pkg: repository FreeBSD has no meta file, using default settings
Fetching packagesite.txz:   0%
pkg: No signature found
pkg: Unable to update repository FreeBSD
root@FriBi:~ #

I was finally able to by-pass the proxy ... and it immediately runned fine ... So I tried to analyze the HTTP dialog. The corporate proxy always sends its responses with Transfer-Encoding: chunked which I suspected to be the cause. I could even confirm it by using a minimal Python proxy that :

    get the response from the corporate proxy with readall() buffering the whole file
    send it back with a ContentLength header to its client (here pkg)

and then again it worked (I could do pkg install xorg ...)

Steps to reproduce :
- use the attached python prog as a proxy with the -C flag to make it chunk its responses
- configure the pkg system to use that proxy
- try to use pkg update -f

Workaround :
- use the attached python prog as a proxy with the -c flag to make it un-chunk its responses

BEWARE : the attached Python file is not thoroughly tested and should not be used except to confirm the bug or as a last hope workaround