Bug 233177

Summary: getting a curl error trying to create a phabricator review
Product: Services Reporter: Mark Johnston <markj>
Component: Code ReviewAssignee: Phabric Admin <phabric-admin>
Status: In Progress ---    
Severity: Affects Many People CC: 0mp, allanjude, cem, dim, gnn, gonzo, jhb, johalun, lwhsu, mat, mqudsi, ngie, pi, sbruno, sunpoet, tcberner
Priority: ---    
Version: unspecified   
Hardware: Any   
OS: Any   

Description Mark Johnston freebsd_committer 2018-11-12 19:13:34 UTC
Linting...                                                    
 LINT OKAY  No lint problems.                                           
Running unit tests...                                                    
No unit test engine is configured for this project.                      
Exception                                                                         
[cURL/92] (https://reviews.freebsd.org/api/differential.creatediff) The cURL library raised an error while making a request. You may be able t
o find more information about this error (error code: 92) on the cURL site: http://curl.haxx.se/libcurl/c/libcurl-errors.html#                
(Run with `--trace` for a full exception trace.)                                                                              
Could not create Phabricator diff               

I tried adding --trace:

...
>>> [36] <exec> $ git log --format=medium -n16 '13e2da91251e0cfa90aa549aded408247af38d17'                                                    
<<< [36] <exec> 5,445 us
>>> [37] <exec> $ git symbolic-ref --quiet HEAD
<<< [37] <exec> 4,262 us
>>> [38] <exec> $ git --version
<<< [38] <exec> 3,429 us
>>> [39] <exec> $ git ls-remote --get-url 'origin'
<<< [39] <exec> 4,122 us
>>> [40] <conduit> repository.query() <bytes = 244>
>>> [41] <http> https://reviews.freebsd.org/api/repository.query
<<< [41] <http> 97,100 us
<<< [40] <conduit> 97,286 us
>>> [42] <conduit> differential.creatediff() <bytes = 109,538>
>>> [43] <http> https://reviews.freebsd.org/api/differential.creatediff                                                                      
<<< [43] <http> 60,092,649 us
<<< [42] <conduit> 60,092,843 us

[2018-11-12 14:12:08] EXCEPTION: (HTTPFutureCURLResponseStatus) [cURL/92] (https://reviews.freebsd.org/api/differential.creatediff) The cURL library raised an error while making a request. You may be able to find more information about this error (error code: 92) on the cURL site: http://curl.haxx.se/libcurl/c/libcurl-errors.html# at [<phutil>/src/future/http/HTTPSFuture.php:408]
arcanist(), phutil()
  #0 HTTPSFuture::isReady() called at [<phutil>/src/future/Future.php:37]
  #1 Future::resolve(NULL) called at [<phutil>/src/future/FutureProxy.php:34]
  #2 FutureProxy::resolve() called at [<phutil>/src/conduit/ConduitClient.php:64]
  #3 ConduitClient::callMethodSynchronous(string, array) called at [<arcanist>/src/workflow/ArcanistDiffWorkflow.php:519]
  #4 ArcanistDiffWorkflow::run() called at [<arcanist>/scripts/arcanist.php:394]
Could not create Phabricator diff
Comment 1 Mark Johnston freebsd_committer 2018-11-13 16:26:38 UTC
Ping?
Comment 2 Mark Johnston freebsd_committer 2018-11-13 16:56:46 UTC
It turned out that I had a relatively ancient client lingering on my workstation: php5-arcanist-20161003.  At some point the client was renamed to arcanist-php* but I guess I never picked that up.  arcanist-php56-20180420_1 exhibits the same problem but the php70 version does not.
Comment 3 Mark Johnston freebsd_committer 2018-11-13 17:40:04 UTC
Sigh, never mind, it's still happening with arcanist-php70-20180420_1.
Comment 4 Allan Jude freebsd_committer 2018-11-13 18:01:27 UTC
We just solved some cURL and SSL problems on the machine that hosts reviews.freebsd.org, does it work now?
Comment 5 Mark Johnston freebsd_committer 2018-11-13 19:25:07 UTC
(In reply to Allan Jude from comment #4)
The problem is still happening, but not consistently.  I just had it fail twice, and then succeed in creating D17981.
Comment 6 Mark Johnston freebsd_committer 2018-11-15 17:28:05 UTC
It's been working reliably for the past day or so, thanks.
Comment 7 Tobias C. Berner freebsd_committer 2018-11-17 09:43:27 UTC
This is is still happening for me.
Comment 8 Mark Johnston freebsd_committer 2018-11-17 19:54:10 UTC
(In reply to Tobias C. Berner from comment #7)
Indeed, the problem has reappeared for me. :(
Comment 9 Sean Bruno freebsd_committer 2018-11-17 20:30:11 UTC
I downrev'd ftp/curl to 1.61.1 in order to work around this.  Since I build my own packages, I did a full php and arcanist reinstall/rebuild.  I'm tagging sunpoet here to get some feedback.
Comment 10 Sunpoet Po-Chuan Hsieh freebsd_committer 2018-11-18 05:31:18 UTC
I just submitted a review [1] successfully.

% pkg info -x curl arcanist libphutil
curl-7.62.0
php72-curl-7.2.12
arcanist-php72-20180420_1
php72-libphutil-20180420

[1] https://reviews.freebsd.org/D18030
Comment 11 Mateusz Piotrowski freebsd_committer 2018-11-25 17:02:32 UTC
(In reply to Sunpoet Po-Chuan Hsieh from comment #10)
Same setup does not work for me.
Comment 12 John Baldwin freebsd_committer freebsd_triage 2018-11-27 01:34:51 UTC
Just failed here:

No unit test engine is configured for this project.
load: 0.83  cmd: php 11021 [running] 68.60r 34.81u 2.69s 100% 69940k
load: 0.85  cmd: php 11021 [running] 69.61r 35.73u 2.73s 100% 69940k
load: 0.85  cmd: php 11021 [running] 70.06r 36.17u 2.76s 99% 69940k
load: 0.85  cmd: php 11021 [running] 70.28r 36.35u 2.77s 100% 69940k
load: 0.85  cmd: php 11021 [running] 70.48r 36.52u 2.79s 100% 69940k
Exception
[cURL/92] (https://reviews.freebsd.org/api/differential.creatediff) The cURL library raised an error while making a request. You may be able to find more information about this error (error code: 92) on the cURL site: http://curl.haxx.se/libcurl/c/libcurl-errors.html#
(Run with `--trace` for a full exception trace.)

(It took a long time to fail, longer than a normal upload FWIW)

curl-7.62.0                    Command line tool and library for transferring data with URLs
php5-arcanist-20161003         Command line interface for Phabricator
php56-curl-5.6.37              The curl shared extension for php
Comment 13 John Baldwin freebsd_committer freebsd_triage 2018-11-27 02:00:11 UTC
So one thing I just noticed is that 'pkg upgrade' appears to not be working correctly on the desktop in question.  My latest poudriere run had built php72-* packages, but 'pkg upgrade -f' kept my old 'php56-*' packages and didn't replace them with the php72 variants.  Maybe this is a bug in how flavors work?
Comment 14 John Baldwin freebsd_committer freebsd_triage 2018-11-27 02:08:40 UTC
So after manually deleting arcanist and using pkg autoremove to rm the old php packages that pkg upgrade failed to fix, I installed the more recent arcanist and it still fails:

arcanist-php72-20180420_1      Command line interface for Phabricator
curl-7.62.0                    Command line tool and library for transferring data with URLs
php72-curl-7.2.12              The curl shared extension for php
Comment 15 Conrad Meyer freebsd_committer 2018-11-27 02:50:00 UTC
(In reply to John Baldwin from comment #14)
Be sure you updated your libphutil as well.  (Yes, it seems like they should have a dependency relation, but no one wants to give me a ports bit and that's probably smart of them.)
Comment 16 John Baldwin freebsd_committer freebsd_triage 2018-11-27 17:10:21 UTC
FYI, I had an updated libhputil.  I tried downgrading just curl without changing any other packages and this fixed arcanist for me:

> sudo pkg install -f ftp/curl
Updating local repository catalogue...
local repository is up to date.
All repositories are up to date.
Checking integrity... done (0 conflicting)
The following 1 package(s) will be affected (of 0 checked):

Installed packages to be DOWNGRADED:
        curl: 7.62.0 -> 7.61.1

Number of packages to be downgraded: 1

Proceed with this action? [y/N]: y
[1/1] Downgrading curl from 7.62.0 to 7.61.1...
[1/1] Extracting curl-7.61.1: 100%

...
Linting...
 LINT OKAY  No lint problems.
Running unit tests...
No unit test engine is configured for this project.
 SKIP STAGING  Unable to determine repository for this change.
Created a new Differential revision:
        Revision URI: https://reviews.freebsd.org/D18360

Included changes:
  M       sys/dev/cxgbe/adapter.h
  M       sys/dev/cxgbe/t4_main.c
  M       sys/dev/cxgbe/t4_netmap.c
  M       sys/dev/cxgbe/t4_sge.c
Comment 17 Sunpoet Po-Chuan Hsieh freebsd_committer 2018-11-27 17:28:59 UTC
Hum, I guess it was caused by CURLE_SSL_CACERT change in curl 7.62.0.

from CHANGES:
  ssl: deprecate CURLE_SSL_CACERT in favour of a unified error code
  Long live CURLE_PEER_FAILED_VERIFICATION

Some of the ports breakage was caught and fixed (r484198, r484199, r484201).
Comment 18 Mateusz Piotrowski freebsd_committer 2018-11-28 15:36:02 UTC
Building curl 7.62.0 without HTTP2 helps (thanks to Rene).
Comment 19 Mathieu Arnold freebsd_committer 2018-12-12 16:11:49 UTC
ping?
Comment 20 George V. Neville-Neil freebsd_committer 2018-12-14 13:32:26 UTC
I've moved all the versions around and still get this on diff --update

 ARGV  '/usr/local/bin/arc' 'diff' '--trace' '--update' 'D18505'
 LOAD  Loaded "phutil" from "/usr/local/lib/php/libphutil/src".
 LOAD  Loaded "arcanist" from "/usr/local/lib/php/arcanist/src".
Config: Reading user configuration file "/usr/home/gnn/.arcrc"...
Config: Did not find system configuration at "/etc/arcconfig".
Working Copy: Reading .arcconfig from "/tank/users/gnn/Repos/svn/FreeBSD.HEAD/.arcconfig".
Working Copy: Path "/tank/users/gnn/Repos/svn/FreeBSD.HEAD" is part of `svn` working copy "/tank/users/gnn/Repos/svn/FreeBSD.HEAD".
Working Copy: Project root is at "/tank/users/gnn/Repos/svn/FreeBSD.HEAD".
Config: Did not find local configuration at "/tank/users/gnn/Repos/svn/FreeBSD.HEAD/.svn/arc/config".
>>> [0] <http> https://reviews.freebsd.org/api/conduit.connect
<<< [0] <http> 223,877 us
>>> [1] <exec> $ svn --xml status
<<< [1] <exec> 1,790,812 us
[2018-12-14 13:31:22] ERROR 2: SimpleXMLElement::__construct(): Entity: line 123: parser error : invalid character in attribute value at [/usr/local/lib/php/arcanist/src/repository/api/ArcanistSubversionAPI.php:78]
arcanist(), phutil()
  #0 SimpleXMLElement::__construct(string) called at [<arcanist>/src/repository/api/ArcanistSubversionAPI.php:78]
  #1 ArcanistSubversionAPI::getSVNStatus() called at [<arcanist>/src/repository/api/ArcanistSubversionAPI.php:54]
  #2 ArcanistSubversionAPI::buildUncommittedStatus() called at [<arcanist>/src/repository/api/ArcanistRepositoryAPI.php:142]
  #3 ArcanistRepositoryAPI::getUncommittedStatus() called at [<arcanist>/src/repository/api/ArcanistRepositoryAPI.php:211]
  #4 ArcanistRepositoryAPI::getUncommittedPathsWithMask(integer) called at [<arcanist>/src/repository/api/ArcanistRepositoryAPI.php:186]
  #5 ArcanistRepositoryAPI::getIncompleteChanges() called at [<arcanist>/src/workflow/ArcanistWorkflow.php:841]
  #6 ArcanistWorkflow::requireCleanWorkingCopy() called at [<arcanist>/src/workflow/ArcanistDiffWorkflow.php:691]
  #7 ArcanistDiffWorkflow::runDiffSetupBasics() called at [<arcanist>/src/workflow/ArcanistDiffWorkflow.php:467]
  #8 ArcanistDiffWorkflow::run() called at [<arcanist>/scripts/arcanist.php:394]
[2018-12-14 13:31:22] ERROR 2: SimpleXMLElement::__construct():    path="tools/build/options/d"> at [/usr/local/lib/php/arcanist/src/repository/api/ArcanistSubversionAPI.php:78]
arcanist(), phutil()
  #0 SimpleXMLElement::__construct(string) called at [<arcanist>/src/repository/api/ArcanistSubversionAPI.php:78]
  #1 ArcanistSubversionAPI::getSVNStatus() called at [<arcanist>/src/repository/api/ArcanistSubversionAPI.php:54]
  #2 ArcanistSubversionAPI::buildUncommittedStatus() called at [<arcanist>/src/repository/api/ArcanistRepositoryAPI.php:142]
  #3 ArcanistRepositoryAPI::getUncommittedStatus() called at [<arcanist>/src/repository/api/ArcanistRepositoryAPI.php:211]
  #4 ArcanistRepositoryAPI::getUncommittedPathsWithMask(integer) called at [<arcanist>/src/repository/api/ArcanistRepositoryAPI.php:186]
  #5 ArcanistRepositoryAPI::getIncompleteChanges() called at [<arcanist>/src/workflow/ArcanistWorkflow.php:841]
  #6 ArcanistWorkflow::requireCleanWorkingCopy() called at [<arcanist>/src/workflow/ArcanistDiffWorkflow.php:691]
  #7 ArcanistDiffWorkflow::runDiffSetupBasics() called at [<arcanist>/src/workflow/ArcanistDiffWorkflow.php:467]
  #8 ArcanistDiffWorkflow::run() called at [<arcanist>/scripts/arcanist.php:394]
[2018-12-14 13:31:22] ERROR 2: SimpleXMLElement::__construct():                               ^ at [/usr/local/lib/php/arcanist/src/repository/api/ArcanistSubversionAPI.php:78]
arcanist(), phutil()
  #0 SimpleXMLElement::__construct(string) called at [<arcanist>/src/repository/api/ArcanistSubversionAPI.php:78]
  #1 ArcanistSubversionAPI::getSVNStatus() called at [<arcanist>/src/repository/api/ArcanistSubversionAPI.php:54]
  #2 ArcanistSubversionAPI::buildUncommittedStatus() called at [<arcanist>/src/repository/api/ArcanistRepositoryAPI.php:142]
  #3 ArcanistRepositoryAPI::getUncommittedStatus() called at [<arcanist>/src/repository/api/ArcanistRepositoryAPI.php:211]
  #4 ArcanistRepositoryAPI::getUncommittedPathsWithMask(integer) called at [<arcanist>/src/repository/api/ArcanistRepositoryAPI.php:186]
  #5 ArcanistRepositoryAPI::getIncompleteChanges() called at [<arcanist>/src/workflow/ArcanistWorkflow.php:841]
  #6 ArcanistWorkflow::requireCleanWorkingCopy() called at [<arcanist>/src/workflow/ArcanistDiffWorkflow.php:691]
  #7 ArcanistDiffWorkflow::runDiffSetupBasics() called at [<arcanist>/src/workflow/ArcanistDiffWorkflow.php:467]
  #8 ArcanistDiffWorkflow::run() called at [<arcanist>/scripts/arcanist.php:394]
[2018-12-14 13:31:22] ERROR 2: SimpleXMLElement::__construct(): Entity: line 123: parser error : attributes construct error at [/usr/local/lib/php/arcanist/src/repository/api/ArcanistSubversionAPI.php:78]
arcanist(), phutil()
  #0 SimpleXMLElement::__construct(string) called at [<arcanist>/src/repository/api/ArcanistSubversionAPI.php:78]
  #1 ArcanistSubversionAPI::getSVNStatus() called at [<arcanist>/src/repository/api/ArcanistSubversionAPI.php:54]
  #2 ArcanistSubversionAPI::buildUncommittedStatus() called at [<arcanist>/src/repository/api/ArcanistRepositoryAPI.php:142]
  #3 ArcanistRepositoryAPI::getUncommittedStatus() called at [<arcanist>/src/repository/api/ArcanistRepositoryAPI.php:211]
  #4 ArcanistRepositoryAPI::getUncommittedPathsWithMask(integer) called at [<arcanist>/src/repository/api/ArcanistRepositoryAPI.php:186]
  #5 ArcanistRepositoryAPI::getIncompleteChanges() called at [<arcanist>/src/workflow/ArcanistWorkflow.php:841]
  #6 ArcanistWorkflow::requireCleanWorkingCopy() called at [<arcanist>/src/workflow/ArcanistDiffWorkflow.php:691]
  #7 ArcanistDiffWorkflow::runDiffSetupBasics() called at [<arcanist>/src/workflow/ArcanistDiffWorkflow.php:467]
  #8 ArcanistDiffWorkflow::run() called at [<arcanist>/scripts/arcanist.php:394]
[2018-12-14 13:31:22] ERROR 2: SimpleXMLElement::__construct():    path="tools/build/options/d"> at [/usr/local/lib/php/arcanist/src/repository/api/ArcanistSubversionAPI.php:78]
arcanist(), phutil()
  #0 SimpleXMLElement::__construct(string) called at [<arcanist>/src/repository/api/ArcanistSubversionAPI.php:78]
  #1 ArcanistSubversionAPI::getSVNStatus() called at [<arcanist>/src/repository/api/ArcanistSubversionAPI.php:54]
  #2 ArcanistSubversionAPI::buildUncommittedStatus() called at [<arcanist>/src/repository/api/ArcanistRepositoryAPI.php:142]
  #3 ArcanistRepositoryAPI::getUncommittedStatus() called at [<arcanist>/src/repository/api/ArcanistRepositoryAPI.php:211]
  #4 ArcanistRepositoryAPI::getUncommittedPathsWithMask(integer) called at [<arcanist>/src/repository/api/ArcanistRepositoryAPI.php:186]
  #5 ArcanistRepositoryAPI::getIncompleteChanges() called at [<arcanist>/src/workflow/ArcanistWorkflow.php:841]
  #6 ArcanistWorkflow::requireCleanWorkingCopy() called at [<arcanist>/src/workflow/ArcanistDiffWorkflow.php:691]
  #7 ArcanistDiffWorkflow::runDiffSetupBasics() called at [<arcanist>/src/workflow/ArcanistDiffWorkflow.php:467]
  #8 ArcanistDiffWorkflow::run() called at [<arcanist>/scripts/arcanist.php:394]
[2018-12-14 13:31:22] ERROR 2: SimpleXMLElement::__construct():                               ^ at [/usr/local/lib/php/arcanist/src/repository/api/ArcanistSubversionAPI.php:78]
arcanist(), phutil()
  #0 SimpleXMLElement::__construct(string) called at [<arcanist>/src/repository/api/ArcanistSubversionAPI.php:78]
  #1 ArcanistSubversionAPI::getSVNStatus() called at [<arcanist>/src/repository/api/ArcanistSubversionAPI.php:54]
  #2 ArcanistSubversionAPI::buildUncommittedStatus() called at [<arcanist>/src/repository/api/ArcanistRepositoryAPI.php:142]
  #3 ArcanistRepositoryAPI::getUncommittedStatus() called at [<arcanist>/src/repository/api/ArcanistRepositoryAPI.php:211]
  #4 ArcanistRepositoryAPI::getUncommittedPathsWithMask(integer) called at [<arcanist>/src/repository/api/ArcanistRepositoryAPI.php:186]
  #5 ArcanistRepositoryAPI::getIncompleteChanges() called at [<arcanist>/src/workflow/ArcanistWorkflow.php:841]
  #6 ArcanistWorkflow::requireCleanWorkingCopy() called at [<arcanist>/src/workflow/ArcanistDiffWorkflow.php:691]
  #7 ArcanistDiffWorkflow::runDiffSetupBasics() called at [<arcanist>/src/workflow/ArcanistDiffWorkflow.php:467]
  #8 ArcanistDiffWorkflow::run() called at [<arcanist>/scripts/arcanist.php:394]
[2018-12-14 13:31:22] ERROR 2: SimpleXMLElement::__construct(): Entity: line 123: parser error : Couldn't find end of Start Tag entry line 122 at [/usr/local/lib/php/arcanist/src/repository/api/ArcanistSubversionAPI.php:78]
arcanist(), phutil()
  #0 SimpleXMLElement::__construct(string) called at [<arcanist>/src/repository/api/ArcanistSubversionAPI.php:78]
  #1 ArcanistSubversionAPI::getSVNStatus() called at [<arcanist>/src/repository/api/ArcanistSubversionAPI.php:54]
  #2 ArcanistSubversionAPI::buildUncommittedStatus() called at [<arcanist>/src/repository/api/ArcanistRepositoryAPI.php:142]
  #3 ArcanistRepositoryAPI::getUncommittedStatus() called at [<arcanist>/src/repository/api/ArcanistRepositoryAPI.php:211]
  #4 ArcanistRepositoryAPI::getUncommittedPathsWithMask(integer) called at [<arcanist>/src/repository/api/ArcanistRepositoryAPI.php:186]
  #5 ArcanistRepositoryAPI::getIncompleteChanges() called at [<arcanist>/src/workflow/ArcanistWorkflow.php:841]
  #6 ArcanistWorkflow::requireCleanWorkingCopy() called at [<arcanist>/src/workflow/ArcanistDiffWorkflow.php:691]
  #7 ArcanistDiffWorkflow::runDiffSetupBasics() called at [<arcanist>/src/workflow/ArcanistDiffWorkflow.php:467]
  #8 ArcanistDiffWorkflow::run() called at [<arcanist>/scripts/arcanist.php:394]
[2018-12-14 13:31:22] ERROR 2: SimpleXMLElement::__construct():    path="tools/build/options/d"> at [/usr/local/lib/php/arcanist/src/repository/api/ArcanistSubversionAPI.php:78]
arcanist(), phutil()
  #0 SimpleXMLElement::__construct(string) called at [<arcanist>/src/repository/api/ArcanistSubversionAPI.php:78]
  #1 ArcanistSubversionAPI::getSVNStatus() called at [<arcanist>/src/repository/api/ArcanistSubversionAPI.php:54]
  #2 ArcanistSubversionAPI::buildUncommittedStatus() called at [<arcanist>/src/repository/api/ArcanistRepositoryAPI.php:142]
  #3 ArcanistRepositoryAPI::getUncommittedStatus() called at [<arcanist>/src/repository/api/ArcanistRepositoryAPI.php:211]
  #4 ArcanistRepositoryAPI::getUncommittedPathsWithMask(integer) called at [<arcanist>/src/repository/api/ArcanistRepositoryAPI.php:186]
  #5 ArcanistRepositoryAPI::getIncompleteChanges() called at [<arcanist>/src/workflow/ArcanistWorkflow.php:841]
  #6 ArcanistWorkflow::requireCleanWorkingCopy() called at [<arcanist>/src/workflow/ArcanistDiffWorkflow.php:691]
  #7 ArcanistDiffWorkflow::runDiffSetupBasics() called at [<arcanist>/src/workflow/ArcanistDiffWorkflow.php:467]
  #8 ArcanistDiffWorkflow::run() called at [<arcanist>/scripts/arcanist.php:394]
[2018-12-14 13:31:22] ERROR 2: SimpleXMLElement::__construct():                               ^ at [/usr/local/lib/php/arcanist/src/repository/api/ArcanistSubversionAPI.php:78]
arcanist(), phutil()
  #0 SimpleXMLElement::__construct(string) called at [<arcanist>/src/repository/api/ArcanistSubversionAPI.php:78]
  #1 ArcanistSubversionAPI::getSVNStatus() called at [<arcanist>/src/repository/api/ArcanistSubversionAPI.php:54]
  #2 ArcanistSubversionAPI::buildUncommittedStatus() called at [<arcanist>/src/repository/api/ArcanistRepositoryAPI.php:142]
  #3 ArcanistRepositoryAPI::getUncommittedStatus() called at [<arcanist>/src/repository/api/ArcanistRepositoryAPI.php:211]
  #4 ArcanistRepositoryAPI::getUncommittedPathsWithMask(integer) called at [<arcanist>/src/repository/api/ArcanistRepositoryAPI.php:186]
  #5 ArcanistRepositoryAPI::getIncompleteChanges() called at [<arcanist>/src/workflow/ArcanistWorkflow.php:841]
  #6 ArcanistWorkflow::requireCleanWorkingCopy() called at [<arcanist>/src/workflow/ArcanistDiffWorkflow.php:691]
  #7 ArcanistDiffWorkflow::runDiffSetupBasics() called at [<arcanist>/src/workflow/ArcanistDiffWorkflow.php:467]
  #8 ArcanistDiffWorkflow::run() called at [<arcanist>/scripts/arcanist.php:394]
[2018-12-14 13:31:22] ERROR 2: SimpleXMLElement::__construct(): Entity: line 123: parser error : PCDATA invalid Char value 7 at [/usr/local/lib/php/arcanist/src/repository/api/ArcanistSubversionAPI.php:78]
arcanist(), phutil()
  #0 SimpleXMLElement::__construct(string) called at [<arcanist>/src/repository/api/ArcanistSubversionAPI.php:78]
  #1 ArcanistSubversionAPI::getSVNStatus() called at [<arcanist>/src/repository/api/ArcanistSubversionAPI.php:54]
  #2 ArcanistSubversionAPI::buildUncommittedStatus() called at [<arcanist>/src/repository/api/ArcanistRepositoryAPI.php:142]
  #3 ArcanistRepositoryAPI::getUncommittedStatus() called at [<arcanist>/src/repository/api/ArcanistRepositoryAPI.php:211]
  #4 ArcanistRepositoryAPI::getUncommittedPathsWithMask(integer) called at [<arcanist>/src/repository/api/ArcanistRepositoryAPI.php:186]
  #5 ArcanistRepositoryAPI::getIncompleteChanges() called at [<arcanist>/src/workflow/ArcanistWorkflow.php:841]
  #6 ArcanistWorkflow::requireCleanWorkingCopy() called at [<arcanist>/src/workflow/ArcanistDiffWorkflow.php:691]
  #7 ArcanistDiffWorkflow::runDiffSetupBasics() called at [<arcanist>/src/workflow/ArcanistDiffWorkflow.php:467]
  #8 ArcanistDiffWorkflow::run() called at [<arcanist>/scripts/arcanist.php:394]
[2018-12-14 13:31:22] ERROR 2: SimpleXMLElement::__construct():    path="tools/build/options/d"> at [/usr/local/lib/php/arcanist/src/repository/api/ArcanistSubversionAPI.php:78]
arcanist(), phutil()
  #0 SimpleXMLElement::__construct(string) called at [<arcanist>/src/repository/api/ArcanistSubversionAPI.php:78]
  #1 ArcanistSubversionAPI::getSVNStatus() called at [<arcanist>/src/repository/api/ArcanistSubversionAPI.php:54]
  #2 ArcanistSubversionAPI::buildUncommittedStatus() called at [<arcanist>/src/repository/api/ArcanistRepositoryAPI.php:142]
  #3 ArcanistRepositoryAPI::getUncommittedStatus() called at [<arcanist>/src/repository/api/ArcanistRepositoryAPI.php:211]
  #4 ArcanistRepositoryAPI::getUncommittedPathsWithMask(integer) called at [<arcanist>/src/repository/api/ArcanistRepositoryAPI.php:186]
  #5 ArcanistRepositoryAPI::getIncompleteChanges() called at [<arcanist>/src/workflow/ArcanistWorkflow.php:841]
  #6 ArcanistWorkflow::requireCleanWorkingCopy() called at [<arcanist>/src/workflow/ArcanistDiffWorkflow.php:691]
  #7 ArcanistDiffWorkflow::runDiffSetupBasics() called at [<arcanist>/src/workflow/ArcanistDiffWorkflow.php:467]
  #8 ArcanistDiffWorkflow::run() called at [<arcanist>/scripts/arcanist.php:394]
[2018-12-14 13:31:22] ERROR 2: SimpleXMLElement::__construct():                               ^ at [/usr/local/lib/php/arcanist/src/repository/api/ArcanistSubversionAPI.php:78]
arcanist(), phutil()
  #0 SimpleXMLElement::__construct(string) called at [<arcanist>/src/repository/api/ArcanistSubversionAPI.php:78]
  #1 ArcanistSubversionAPI::getSVNStatus() called at [<arcanist>/src/repository/api/ArcanistSubversionAPI.php:54]
  #2 ArcanistSubversionAPI::buildUncommittedStatus() called at [<arcanist>/src/repository/api/ArcanistRepositoryAPI.php:142]
  #3 ArcanistRepositoryAPI::getUncommittedStatus() called at [<arcanist>/src/repository/api/ArcanistRepositoryAPI.php:211]
  #4 ArcanistRepositoryAPI::getUncommittedPathsWithMask(integer) called at [<arcanist>/src/repository/api/ArcanistRepositoryAPI.php:186]
  #5 ArcanistRepositoryAPI::getIncompleteChanges() called at [<arcanist>/src/workflow/ArcanistWorkflow.php:841]
  #6 ArcanistWorkflow::requireCleanWorkingCopy() called at [<arcanist>/src/workflow/ArcanistDiffWorkflow.php:691]
  #7 ArcanistDiffWorkflow::runDiffSetupBasics() called at [<arcanist>/src/workflow/ArcanistDiffWorkflow.php:467]
  #8 ArcanistDiffWorkflow::run() called at [<arcanist>/scripts/arcanist.php:394]
[2018-12-14 13:31:22] ERROR 2: SimpleXMLElement::__construct(): Entity: line 123: parser error : PCDATA invalid Char value 7 at [/usr/local/lib/php/arcanist/src/repository/api/ArcanistSubversionAPI.php:78]
arcanist(), phutil()
  #0 SimpleXMLElement::__construct(string) called at [<arcanist>/src/repository/api/ArcanistSubversionAPI.php:78]
  #1 ArcanistSubversionAPI::getSVNStatus() called at [<arcanist>/src/repository/api/ArcanistSubversionAPI.php:54]
  #2 ArcanistSubversionAPI::buildUncommittedStatus() called at [<arcanist>/src/repository/api/ArcanistRepositoryAPI.php:142]
  #3 ArcanistRepositoryAPI::getUncommittedStatus() called at [<arcanist>/src/repository/api/ArcanistRepositoryAPI.php:211]
  #4 ArcanistRepositoryAPI::getUncommittedPathsWithMask(integer) called at [<arcanist>/src/repository/api/ArcanistRepositoryAPI.php:186]
  #5 ArcanistRepositoryAPI::getIncompleteChanges() called at [<arcanist>/src/workflow/ArcanistWorkflow.php:841]
  #6 ArcanistWorkflow::requireCleanWorkingCopy() called at [<arcanist>/src/workflow/ArcanistDiffWorkflow.php:691]
  #7 ArcanistDiffWorkflow::runDiffSetupBasics() called at [<arcanist>/src/workflow/ArcanistDiffWorkflow.php:467]
  #8 ArcanistDiffWorkflow::run() called at [<arcanist>/scripts/arcanist.php:394]
[2018-12-14 13:31:22] ERROR 2: SimpleXMLElement::__construct():    path="tools/build/options/d"> at [/usr/local/lib/php/arcanist/src/repository/api/ArcanistSubversionAPI.php:78]
arcanist(), phutil()
  #0 SimpleXMLElement::__construct(string) called at [<arcanist>/src/repository/api/ArcanistSubversionAPI.php:78]
  #1 ArcanistSubversionAPI::getSVNStatus() called at [<arcanist>/src/repository/api/ArcanistSubversionAPI.php:54]
  #2 ArcanistSubversionAPI::buildUncommittedStatus() called at [<arcanist>/src/repository/api/ArcanistRepositoryAPI.php:142]
  #3 ArcanistRepositoryAPI::getUncommittedStatus() called at [<arcanist>/src/repository/api/ArcanistRepositoryAPI.php:211]
  #4 ArcanistRepositoryAPI::getUncommittedPathsWithMask(integer) called at [<arcanist>/src/repository/api/ArcanistRepositoryAPI.php:186]
  #5 ArcanistRepositoryAPI::getIncompleteChanges() called at [<arcanist>/src/workflow/ArcanistWorkflow.php:841]
  #6 ArcanistWorkflow::requireCleanWorkingCopy() called at [<arcanist>/src/workflow/ArcanistDiffWorkflow.php:691]
  #7 ArcanistDiffWorkflow::runDiffSetupBasics() called at [<arcanist>/src/workflow/ArcanistDiffWorkflow.php:467]
  #8 ArcanistDiffWorkflow::run() called at [<arcanist>/scripts/arcanist.php:394]
[2018-12-14 13:31:22] ERROR 2: SimpleXMLElement::__construct():                                ^ at [/usr/local/lib/php/arcanist/src/repository/api/ArcanistSubversionAPI.php:78]
arcanist(), phutil()
  #0 SimpleXMLElement::__construct(string) called at [<arcanist>/src/repository/api/ArcanistSubversionAPI.php:78]
  #1 ArcanistSubversionAPI::getSVNStatus() called at [<arcanist>/src/repository/api/ArcanistSubversionAPI.php:54]
  #2 ArcanistSubversionAPI::buildUncommittedStatus() called at [<arcanist>/src/repository/api/ArcanistRepositoryAPI.php:142]
  #3 ArcanistRepositoryAPI::getUncommittedStatus() called at [<arcanist>/src/repository/api/ArcanistRepositoryAPI.php:211]
  #4 ArcanistRepositoryAPI::getUncommittedPathsWithMask(integer) called at [<arcanist>/src/repository/api/ArcanistRepositoryAPI.php:186]
  #5 ArcanistRepositoryAPI::getIncompleteChanges() called at [<arcanist>/src/workflow/ArcanistWorkflow.php:841]
  #6 ArcanistWorkflow::requireCleanWorkingCopy() called at [<arcanist>/src/workflow/ArcanistDiffWorkflow.php:691]
  #7 ArcanistDiffWorkflow::runDiffSetupBasics() called at [<arcanist>/src/workflow/ArcanistDiffWorkflow.php:467]
  #8 ArcanistDiffWorkflow::run() called at [<arcanist>/scripts/arcanist.php:394]
[2018-12-14 13:31:22] ERROR 2: SimpleXMLElement::__construct(): Entity: line 128: parser error : Opening and ending tag mismatch: target line 3 and entry at [/usr/local/lib/php/arcanist/src/repository/api/ArcanistSubversionAPI.php:78]
arcanist(), phutil()
  #0 SimpleXMLElement::__construct(string) called at [<arcanist>/src/repository/api/ArcanistSubversionAPI.php:78]
  #1 ArcanistSubversionAPI::getSVNStatus() called at [<arcanist>/src/repository/api/ArcanistSubversionAPI.php:54]
  #2 ArcanistSubversionAPI::buildUncommittedStatus() called at [<arcanist>/src/repository/api/ArcanistRepositoryAPI.php:142]
  #3 ArcanistRepositoryAPI::getUncommittedStatus() called at [<arcanist>/src/repository/api/ArcanistRepositoryAPI.php:211]
  #4 ArcanistRepositoryAPI::getUncommittedPathsWithMask(integer) called at [<arcanist>/src/repository/api/ArcanistRepositoryAPI.php:186]
  #5 ArcanistRepositoryAPI::getIncompleteChanges() called at [<arcanist>/src/workflow/ArcanistWorkflow.php:841]
  #6 ArcanistWorkflow::requireCleanWorkingCopy() called at [<arcanist>/src/workflow/ArcanistDiffWorkflow.php:691]
  #7 ArcanistDiffWorkflow::runDiffSetupBasics() called at [<arcanist>/src/workflow/ArcanistDiffWorkflow.php:467]
  #8 ArcanistDiffWorkflow::run() called at [<arcanist>/scripts/arcanist.php:394]
[2018-12-14 13:31:22] ERROR 2: SimpleXMLElement::__construct(): </entry> at [/usr/local/lib/php/arcanist/src/repository/api/ArcanistSubversionAPI.php:78]
arcanist(), phutil()
  #0 SimpleXMLElement::__construct(string) called at [<arcanist>/src/repository/api/ArcanistSubversionAPI.php:78]
  #1 ArcanistSubversionAPI::getSVNStatus() called at [<arcanist>/src/repository/api/ArcanistSubversionAPI.php:54]
  #2 ArcanistSubversionAPI::buildUncommittedStatus() called at [<arcanist>/src/repository/api/ArcanistRepositoryAPI.php:142]
  #3 ArcanistRepositoryAPI::getUncommittedStatus() called at [<arcanist>/src/repository/api/ArcanistRepositoryAPI.php:211]
  #4 ArcanistRepositoryAPI::getUncommittedPathsWithMask(integer) called at [<arcanist>/src/repository/api/ArcanistRepositoryAPI.php:186]
  #5 ArcanistRepositoryAPI::getIncompleteChanges() called at [<arcanist>/src/workflow/ArcanistWorkflow.php:841]
  #6 ArcanistWorkflow::requireCleanWorkingCopy() called at [<arcanist>/src/workflow/ArcanistDiffWorkflow.php:691]
  #7 ArcanistDiffWorkflow::runDiffSetupBasics() called at [<arcanist>/src/workflow/ArcanistDiffWorkflow.php:467]
  #8 ArcanistDiffWorkflow::run() called at [<arcanist>/scripts/arcanist.php:394]
[2018-12-14 13:31:22] ERROR 2: SimpleXMLElement::__construct():         ^ at [/usr/local/lib/php/arcanist/src/repository/api/ArcanistSubversionAPI.php:78]
arcanist(), phutil()
  #0 SimpleXMLElement::__construct(string) called at [<arcanist>/src/repository/api/ArcanistSubversionAPI.php:78]
  #1 ArcanistSubversionAPI::getSVNStatus() called at [<arcanist>/src/repository/api/ArcanistSubversionAPI.php:54]
  #2 ArcanistSubversionAPI::buildUncommittedStatus() called at [<arcanist>/src/repository/api/ArcanistRepositoryAPI.php:142]
  #3 ArcanistRepositoryAPI::getUncommittedStatus() called at [<arcanist>/src/repository/api/ArcanistRepositoryAPI.php:211]
  #4 ArcanistRepositoryAPI::getUncommittedPathsWithMask(integer) called at [<arcanist>/src/repository/api/ArcanistRepositoryAPI.php:186]
  #5 ArcanistRepositoryAPI::getIncompleteChanges() called at [<arcanist>/src/workflow/ArcanistWorkflow.php:841]
  #6 ArcanistWorkflow::requireCleanWorkingCopy() called at [<arcanist>/src/workflow/ArcanistDiffWorkflow.php:691]
  #7 ArcanistDiffWorkflow::runDiffSetupBasics() called at [<arcanist>/src/workflow/ArcanistDiffWorkflow.php:467]
  #8 ArcanistDiffWorkflow::run() called at [<arcanist>/scripts/arcanist.php:394]
[2018-12-14 13:31:22] ERROR 2: SimpleXMLElement::__construct(): Entity: line 162: parser error : Opening and ending tag mismatch: status line 2 and target at [/usr/local/lib/php/arcanist/src/repository/api/ArcanistSubversionAPI.php:78]
arcanist(), phutil()
  #0 SimpleXMLElement::__construct(string) called at [<arcanist>/src/repository/api/ArcanistSubversionAPI.php:78]
  #1 ArcanistSubversionAPI::getSVNStatus() called at [<arcanist>/src/repository/api/ArcanistSubversionAPI.php:54]
  #2 ArcanistSubversionAPI::buildUncommittedStatus() called at [<arcanist>/src/repository/api/ArcanistRepositoryAPI.php:142]
  #3 ArcanistRepositoryAPI::getUncommittedStatus() called at [<arcanist>/src/repository/api/ArcanistRepositoryAPI.php:211]
  #4 ArcanistRepositoryAPI::getUncommittedPathsWithMask(integer) called at [<arcanist>/src/repository/api/ArcanistRepositoryAPI.php:186]
  #5 ArcanistRepositoryAPI::getIncompleteChanges() called at [<arcanist>/src/workflow/ArcanistWorkflow.php:841]
  #6 ArcanistWorkflow::requireCleanWorkingCopy() called at [<arcanist>/src/workflow/ArcanistDiffWorkflow.php:691]
  #7 ArcanistDiffWorkflow::runDiffSetupBasics() called at [<arcanist>/src/workflow/ArcanistDiffWorkflow.php:467]
  #8 ArcanistDiffWorkflow::run() called at [<arcanist>/scripts/arcanist.php:394]
[2018-12-14 13:31:22] ERROR 2: SimpleXMLElement::__construct(): </target> at [/usr/local/lib/php/arcanist/src/repository/api/ArcanistSubversionAPI.php:78]
arcanist(), phutil()
  #0 SimpleXMLElement::__construct(string) called at [<arcanist>/src/repository/api/ArcanistSubversionAPI.php:78]
  #1 ArcanistSubversionAPI::getSVNStatus() called at [<arcanist>/src/repository/api/ArcanistSubversionAPI.php:54]
  #2 ArcanistSubversionAPI::buildUncommittedStatus() called at [<arcanist>/src/repository/api/ArcanistRepositoryAPI.php:142]
  #3 ArcanistRepositoryAPI::getUncommittedStatus() called at [<arcanist>/src/repository/api/ArcanistRepositoryAPI.php:211]
  #4 ArcanistRepositoryAPI::getUncommittedPathsWithMask(integer) called at [<arcanist>/src/repository/api/ArcanistRepositoryAPI.php:186]
  #5 ArcanistRepositoryAPI::getIncompleteChanges() called at [<arcanist>/src/workflow/ArcanistWorkflow.php:841]
  #6 ArcanistWorkflow::requireCleanWorkingCopy() called at [<arcanist>/src/workflow/ArcanistDiffWorkflow.php:691]
  #7 ArcanistDiffWorkflow::runDiffSetupBasics() called at [<arcanist>/src/workflow/ArcanistDiffWorkflow.php:467]
  #8 ArcanistDiffWorkflow::run() called at [<arcanist>/scripts/arcanist.php:394]
[2018-12-14 13:31:22] ERROR 2: SimpleXMLElement::__construct():          ^ at [/usr/local/lib/php/arcanist/src/repository/api/ArcanistSubversionAPI.php:78]
arcanist(), phutil()
  #0 SimpleXMLElement::__construct(string) called at [<arcanist>/src/repository/api/ArcanistSubversionAPI.php:78]
  #1 ArcanistSubversionAPI::getSVNStatus() called at [<arcanist>/src/repository/api/ArcanistSubversionAPI.php:54]
  #2 ArcanistSubversionAPI::buildUncommittedStatus() called at [<arcanist>/src/repository/api/ArcanistRepositoryAPI.php:142]
  #3 ArcanistRepositoryAPI::getUncommittedStatus() called at [<arcanist>/src/repository/api/ArcanistRepositoryAPI.php:211]
  #4 ArcanistRepositoryAPI::getUncommittedPathsWithMask(integer) called at [<arcanist>/src/repository/api/ArcanistRepositoryAPI.php:186]
  #5 ArcanistRepositoryAPI::getIncompleteChanges() called at [<arcanist>/src/workflow/ArcanistWorkflow.php:841]
  #6 ArcanistWorkflow::requireCleanWorkingCopy() called at [<arcanist>/src/workflow/ArcanistDiffWorkflow.php:691]
  #7 ArcanistDiffWorkflow::runDiffSetupBasics() called at [<arcanist>/src/workflow/ArcanistDiffWorkflow.php:467]
  #8 ArcanistDiffWorkflow::run() called at [<arcanist>/scripts/arcanist.php:394]
[2018-12-14 13:31:22] ERROR 2: SimpleXMLElement::__construct(): Entity: line 163: parser error : Extra content at the end of the document at [/usr/local/lib/php/arcanist/src/repository/api/ArcanistSubversionAPI.php:78]
arcanist(), phutil()
  #0 SimpleXMLElement::__construct(string) called at [<arcanist>/src/repository/api/ArcanistSubversionAPI.php:78]
  #1 ArcanistSubversionAPI::getSVNStatus() called at [<arcanist>/src/repository/api/ArcanistSubversionAPI.php:54]
  #2 ArcanistSubversionAPI::buildUncommittedStatus() called at [<arcanist>/src/repository/api/ArcanistRepositoryAPI.php:142]
  #3 ArcanistRepositoryAPI::getUncommittedStatus() called at [<arcanist>/src/repository/api/ArcanistRepositoryAPI.php:211]
  #4 ArcanistRepositoryAPI::getUncommittedPathsWithMask(integer) called at [<arcanist>/src/repository/api/ArcanistRepositoryAPI.php:186]
  #5 ArcanistRepositoryAPI::getIncompleteChanges() called at [<arcanist>/src/workflow/ArcanistWorkflow.php:841]
  #6 ArcanistWorkflow::requireCleanWorkingCopy() called at [<arcanist>/src/workflow/ArcanistDiffWorkflow.php:691]
  #7 ArcanistDiffWorkflow::runDiffSetupBasics() called at [<arcanist>/src/workflow/ArcanistDiffWorkflow.php:467]
  #8 ArcanistDiffWorkflow::run() called at [<arcanist>/scripts/arcanist.php:394]
[2018-12-14 13:31:22] ERROR 2: SimpleXMLElement::__construct(): </status> at [/usr/local/lib/php/arcanist/src/repository/api/ArcanistSubversionAPI.php:78]
arcanist(), phutil()
  #0 SimpleXMLElement::__construct(string) called at [<arcanist>/src/repository/api/ArcanistSubversionAPI.php:78]
  #1 ArcanistSubversionAPI::getSVNStatus() called at [<arcanist>/src/repository/api/ArcanistSubversionAPI.php:54]
  #2 ArcanistSubversionAPI::buildUncommittedStatus() called at [<arcanist>/src/repository/api/ArcanistRepositoryAPI.php:142]
  #3 ArcanistRepositoryAPI::getUncommittedStatus() called at [<arcanist>/src/repository/api/ArcanistRepositoryAPI.php:211]
  #4 ArcanistRepositoryAPI::getUncommittedPathsWithMask(integer) called at [<arcanist>/src/repository/api/ArcanistRepositoryAPI.php:186]
  #5 ArcanistRepositoryAPI::getIncompleteChanges() called at [<arcanist>/src/workflow/ArcanistWorkflow.php:841]
  #6 ArcanistWorkflow::requireCleanWorkingCopy() called at [<arcanist>/src/workflow/ArcanistDiffWorkflow.php:691]
  #7 ArcanistDiffWorkflow::runDiffSetupBasics() called at [<arcanist>/src/workflow/ArcanistDiffWorkflow.php:467]
  #8 ArcanistDiffWorkflow::run() called at [<arcanist>/scripts/arcanist.php:394]
[2018-12-14 13:31:22] ERROR 2: SimpleXMLElement::__construct(): ^ at [/usr/local/lib/php/arcanist/src/repository/api/ArcanistSubversionAPI.php:78]
arcanist(), phutil()
  #0 SimpleXMLElement::__construct(string) called at [<arcanist>/src/repository/api/ArcanistSubversionAPI.php:78]
  #1 ArcanistSubversionAPI::getSVNStatus() called at [<arcanist>/src/repository/api/ArcanistSubversionAPI.php:54]
  #2 ArcanistSubversionAPI::buildUncommittedStatus() called at [<arcanist>/src/repository/api/ArcanistRepositoryAPI.php:142]
  #3 ArcanistRepositoryAPI::getUncommittedStatus() called at [<arcanist>/src/repository/api/ArcanistRepositoryAPI.php:211]
  #4 ArcanistRepositoryAPI::getUncommittedPathsWithMask(integer) called at [<arcanist>/src/repository/api/ArcanistRepositoryAPI.php:186]
  #5 ArcanistRepositoryAPI::getIncompleteChanges() called at [<arcanist>/src/workflow/ArcanistWorkflow.php:841]
  #6 ArcanistWorkflow::requireCleanWorkingCopy() called at [<arcanist>/src/workflow/ArcanistDiffWorkflow.php:691]
  #7 ArcanistDiffWorkflow::runDiffSetupBasics() called at [<arcanist>/src/workflow/ArcanistDiffWorkflow.php:467]
  #8 ArcanistDiffWorkflow::run() called at [<arcanist>/scripts/arcanist.php:394]

[2018-12-14 13:31:22] EXCEPTION: (Exception) String could not be parsed as XML at [<arcanist>/src/repository/api/ArcanistSubversionAPI.php:78]
arcanist(), phutil()
  #0 SimpleXMLElement::__construct(string) called at [<arcanist>/src/repository/api/ArcanistSubversionAPI.php:78]
  #1 ArcanistSubversionAPI::getSVNStatus() called at [<arcanist>/src/repository/api/ArcanistSubversionAPI.php:54]
  #2 ArcanistSubversionAPI::buildUncommittedStatus() called at [<arcanist>/src/repository/api/ArcanistRepositoryAPI.php:142]
  #3 ArcanistRepositoryAPI::getUncommittedStatus() called at [<arcanist>/src/repository/api/ArcanistRepositoryAPI.php:211]
  #4 ArcanistRepositoryAPI::getUncommittedPathsWithMask(integer) called at [<arcanist>/src/repository/api/ArcanistRepositoryAPI.php:186]
  #5 ArcanistRepositoryAPI::getIncompleteChanges() called at [<arcanist>/src/workflow/ArcanistWorkflow.php:841]
  #6 ArcanistWorkflow::requireCleanWorkingCopy() called at [<arcanist>/src/workflow/ArcanistDiffWorkflow.php:691]
  #7 ArcanistDiffWorkflow::runDiffSetupBasics() called at [<arcanist>/src/workflow/ArcanistDiffWorkflow.php:467]
  #8 ArcanistDiffWorkflow::run() called at [<arcanist>/scripts/arcanist.php:394]
Comment 21 Li-Wen Hsu freebsd_committer 2018-12-14 15:24:46 UTC
After discussing with sunpoet, we found that updating libnghttp2 1.35.1 might help (although the changelog seems not related). I've updated it in https://svnweb.freebsd.org/changeset/ports/487439 and it works for me to create/update differential on https://reviews.freebsd.org/ and https://reviews-dev.freebsd.org/.

My other packages from pkg.freebsd.org:
arcanist-php72-20180420_1
curl-7.62.0
php72-curl-7.2.13
Comment 22 Mathieu Arnold freebsd_committer 2018-12-19 14:15:37 UTC
I can't find out where I read that, but someone wrote that upgrading libnghttp2 to its latest version fixed it, well, it does not :(
Comment 23 Mathieu Arnold freebsd_committer 2018-12-19 14:16:55 UTC
Oh, it was in comment #21 just above mine, never mind, it does not work for me :(
Comment 24 Li-Wen Hsu freebsd_committer 2018-12-19 14:18:20 UTC
(In reply to Mathieu Arnold from comment #22)
mat, what's your installed packages and the error message?
Comment 25 Dimitry Andric freebsd_committer 2018-12-30 19:28:26 UTC
Another "me too". Relevant package versions:

arcanist-php72-20180420_1
curl-7.62.0
libnghttp2-1.35.1
php72-7.2.13
php72-curl-7.2.13
php72-dom-7.2.13
php72-hash-7.2.13
php72-json-7.2.13
php72-libphutil-20180420
php72-mbstring-7.2.13
php72-simplexml-7.2.13
php72-xml-7.2.13

I captured the http2 traffic with tcpdump, and decoded it in Wireshark (the 2001: machine is the client, 2610: is the server):

https://www.andric.com/freebsd/phab1.png

As you can see, the connection stalls for about 60 seconds, between packet 81 and 124, after which the server sends a "RST_STREAM[9]" message, which is probably the cause for the curl error.

Maybe the WINDOW_UPDATE[9] message in packet 81 is what is causing the problem?

I guess it is difficult to debug this further without more logging on the server side.
Comment 26 Sean Bruno freebsd_committer 2019-01-02 21:50:33 UTC
Does disabling HTTP2 support in ftp/curl resolve this for anyone?
Comment 27 Dimitry Andric freebsd_committer 2019-01-03 11:38:11 UTC
(In reply to Sean Bruno from comment #26)
> Does disabling HTTP2 support in ftp/curl resolve this for anyone?

Yes, for me it does, at least.
Comment 28 Sean Bruno freebsd_committer 2019-01-03 14:37:31 UTC
(In reply to Dimitry Andric from comment #27)
Reported to the curl-users list as we don't seem to have any obvious fix in our universe.

https://curl.haxx.se/mail/archive-2019-01/0004.html
Comment 29 Sean Bruno freebsd_committer 2019-01-03 15:53:30 UTC
Does anyone have a one-liner, with curl preferably, that explodes so that we can have the curl folks look at it?  Everything I tried was rejected or got an invalid session error.
Comment 30 Mahmoud Al-Qudsi 2019-01-24 18:27:43 UTC
This is still completely broken for me (no matter how many times I try). Just a thought, but could it have anything to do with whether the request is made over IPv4 or IPv6?

At any rate, the following patch to arcanist/libphutil works around the problem for me:


```diff
*** HTTPSFuture.php     Thu Jan 24 12:16:40 2019
--- /usr/local/lib/php/libphutil/src/future/http/HTTPSFuture.php        Thu Jan 24 12:13:43 2019
***************
*** 204,213 ****
--- 204,214 ----

        $this->handle = $curl;
        curl_multi_add_handle(self::$multi, $curl);

        curl_setopt($curl, CURLOPT_URL, $uri);
+       curl_setopt($curl, CURLOPT_HTTP_VERSION, CURL_HTTP_VERSION_1_1);

        if (defined('CURLOPT_PROTOCOLS')) {
          // cURL supports a lot of protocols, and by default it will honor
          // redirects across protocols (for instance, from HTTP to POP3). Beyond
          // being very silly, this also has security implications:
```
Comment 31 Sunpoet Po-Chuan Hsieh freebsd_committer 2019-01-24 19:21:25 UTC
I just submitted a review [1] using curl with HTTP2 enabled.

% uname -v
FreeBSD 11.2-RELEASE #0 r335510: Fri Jun 22 04:32:14 UTC 2018     root@releng2.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC
% pkg info -x arcanist curl nghttp2
arcanist-php73-20181231_1
curl-7.63.0_1
php73-curl-7.3.1_1
libnghttp2-1.36.0

[1] https://reviews.freebsd.org/D18945
Comment 32 Conrad Meyer freebsd_committer 2019-02-05 20:32:06 UTC
I had to use the comment #30 workaround to submit https://reviews.freebsd.org/D19088 with curl 7.63.  Failed before, no problem after restricting to HTTP_1_1.
Comment 33 Johannes Lundberg freebsd_committer 2019-02-07 11:10:50 UTC
Same here. After multiple failures, it works right away when forcing http 1.1 as described in #30.
Comment 34 John Baldwin freebsd_committer freebsd_triage 2019-02-08 19:08:48 UTC
Just tripped over this yet again on a new laptop.  The hack from comment #30 fixed it for me.

Can we please try disabling http/2 on the server side again?  I can easily retest by commenting out the one line hack.
Comment 35 Sean Bruno freebsd_committer 2019-02-08 23:41:31 UTC
(In reply to John Baldwin from comment #34)
I've totally disabled HTTP2 for all server configurations.  I can verify that Firefox detects reviews.freebsd.org as HTTP1.1 now.

Give it a whirl and see if this is just the way things need to be now.
Comment 36 John Baldwin freebsd_committer freebsd_triage 2019-02-08 23:54:38 UTC
(In reply to Sean Bruno from comment #35)
I removed the hack from comment #30 and was able to upload an updated diff to an existing revision just fine.  Perhaps it would be good to respond to the thread on developers@ so some other folks can confirm it works for them now?
Comment 37 Enji Cooper freebsd_committer 2019-04-13 06:53:32 UTC
I tried updating my revision, like so, but the diff update is timing out server side when HTTP/2 was enabled:

```
$ arc diff --trace --update D19851 lib/libpmc
...
>>> [34] (+4,161) <http> https://reviews.freebsd.org/api/repository.query
<<< [34] (+4,333) <http> 171,407 us
>>> [35] (+4,333) <http> https://reviews.freebsd.org/api/differential.creatediff
<<< [35] (+64,790) <http> 60,456,562 us

[2019-04-13 06:45:12] EXCEPTION: (HTTPFutureCURLResponseStatus) [cURL/92] (https://reviews.freebsd.org/api/differential.creatediff) The cURL library raised an error while making a request. You may be able to find more information about this error (error code: 92) on the cURL site: http://curl.haxx.se/libcurl/c/libcurl-errors.html# at [<phutil>/src/future/http/HTTPSFuture.php:408]
arcanist(), phutil()
  #0 HTTPSFuture::isReady() called at [<phutil>/src/future/Future.php:37]
  #1 Future::resolve(NULL) called at [<phutil>/src/future/FutureProxy.php:34]
  #2 FutureProxy::resolve() called at [<phutil>/src/conduit/ConduitClient.php:64]
  #3 ConduitClient::callMethodSynchronous(string, array) called at [<arcanist>/src/workflow/ArcanistDiffWorkflow.php:518]
  #4 ArcanistDiffWorkflow::run() called at [<arcanist>/scripts/arcanist.php:394]
```

Looking at the curl error, it’s "CURLE_HTTP2_STREAM (92)"; the description for the error code is: "Stream error in the HTTP/2 framing layer.”

When I applied the hack noted in #30, I was able to submit a review update.
Comment 38 Enji Cooper freebsd_committer 2019-04-13 07:17:44 UTC
Looking at this StackOverflow post, I set `CURLOPT_VERBOSE` and got this "noise":

* Found bundle for host reviews.freebsd.org: 0x806631180 [can multiplex]
* Re-using existing connection! (#0) with host reviews.freebsd.org
* Connected to reviews.freebsd.org (96.47.72.197) port 443 (#0)
* Using Stream ID: 9 (easy handle 0x806647800)
> POST /api/repository.query HTTP/2
Host: reviews.freebsd.org
Accept: */*
Content-Length: 182
Content-Type: application/x-www-form-urlencoded

* We are completely uploaded and fine
< HTTP/2 200 
< date: Sat, 13 Apr 2019 07:01:25 GMT
< content-type: application/json
< set-cookie: phsid=A%2F7qwuffh5foopwfiuaho62feqk3jsmsyfxslgkis2; expires=Thu, 11-Apr-2024 07:01:25 GMT; Max-Age=157680000; path=/; domain=reviews.freebsd.org; secure; httponly
< x-frame-options: Deny
< strict-transport-security: max-age=31536000; includeSubDomains
< content-security-policy: default-src 'self' https://reviews.freebsd.org/ 'unsafe-inline' wss://reviews.freebsd.org/ws/ https://www.google.com/
< referrer-policy: no-referrer
< cache-control: no-store
< expires: Sat, 01 Jan 2000 00:00:00 GMT
< x-content-type-options: nosniff
< server: CloudSoft_Enterprise/v0.01
< x-xss-protection: 1; mode=block
< 
* Connection #0 to host reviews.freebsd.org left intact
<<< [34] (+19,106) <http> 167,505 us
>>> [35] (+19,107) <http> https://reviews.freebsd.org/api/differential.creatediff
* Found bundle for host reviews.freebsd.org: 0x806631180 [can multiplex]
* Re-using existing connection! (#0) with host reviews.freebsd.org
* Connected to reviews.freebsd.org (96.47.72.197) port 443 (#0)
* Using Stream ID: b (easy handle 0x806647800)
> POST /api/differential.creatediff HTTP/2
Host: reviews.freebsd.org
Accept: */*
Content-Length: 79897
Content-Type: application/x-www-form-urlencoded

* We are completely uploaded and fine
* HTTP/2 stream 0 was not closed cleanly: PROTOCOL_ERROR (err 1)
* stopped the pause stream!
* Connection #0 to host reviews.freebsd.org left intact
<<< [35] (+79,551) <http> 60,444,163 us

[2019-04-13 07:03:23] EXCEPTION: (HTTPFutureCURLResponseStatus) [cURL/92] (https://reviews.freebsd.org/api/differential.creatediff) The cURL library raised an error while making a request. You may be able to find more information about this error (error code: 92) on the cURL site: http://curl.haxx.se/libcurl/c/libcurl-errors.html# at [<phutil>/src/future/http/HTTPSFuture.php:410]
arcanist(), phutil()
  #0 HTTPSFuture::isReady() called at [<phutil>/src/future/Future.php:37]
  #1 Future::resolve(NULL) called at [<phutil>/src/future/FutureProxy.php:34]
  #2 FutureProxy::resolve() called at [<phutil>/src/conduit/ConduitClient.php:64]
  #3 ConduitClient::callMethodSynchronous(string, array) called at [<arcanist>/src/workflow/ArcanistDiffWorkflow.php:518]
  #4 ArcanistDiffWorkflow::run() called at [<arcanist>/scripts/arcanist.php:394]
* Closing connection 0

Ok, here's a fun datapoint: this issue was introduced between curl 7.60.0 and 7.64.1. When I installed 7.60.0 from the release package, it worked perfectly fine running `arc diff --update`.
Comment 39 Enji Cooper freebsd_committer 2019-04-13 07:23:28 UTC
Oy... this issue looks like a winner: https://github.com/curl/curl/issues/3570 .

Do we use HTTP(S)? proxies? https://curl.haxx.se/docs/todo.html#Support_HTTP_2_for_HTTP_S_proxi

I'll try pulling down the patch from https://github.com/curl/curl/issues/3570 and see if things work.
Comment 40 Enji Cooper freebsd_committer 2019-04-13 07:57:47 UTC
(In reply to Enji Cooper from comment #39)

Narrowed down the issue to curl-7.61.0 and curl-7.63.0.
Comment 41 Enji Cooper freebsd_committer 2019-04-13 08:00:53 UTC
(In reply to Enji Cooper from comment #40)

Ok, I've found a working version (7.61.1) and a broken version (7.62.0). Digging through the changeset...
Comment 42 Enji Cooper freebsd_committer 2019-04-13 08:24:02 UTC
Filed an upstream libcurl issue: https://github.com/curl/curl/issues/3775 .
Comment 43 Conrad Meyer freebsd_committer 2019-04-13 14:27:26 UTC
(In reply to Enji Cooper from comment #38)
> Ok, here's a fun datapoint: this issue was introduced between curl 7.60.0
> and 7.64.1. When I installed 7.60.0 from the release package, it worked
> perfectly fine running `arc diff --update`.

Right, we had narrowed it down to 7.62 (from 7.61.1) in https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=233177#c16 and offline (IRC).
Comment 44 Enji Cooper freebsd_committer 2019-04-13 15:11:53 UTC
(In reply to Conrad Meyer from comment #43)

Ah, I missed that part and didn’t go through the complete bug history.
Comment 45 Mahmoud Al-Qudsi 2019-04-13 15:21:53 UTC
There aren't an overwhelming number of comments, you probably spent more time chasing this down than it would have taken to read through them. Also not a bad idea to group your thoughts then submit a single, cohesive comment so future readers don't need to read through possible dead-ends or a wall of minutia.