Bug 148515 - Memory / syslog strangeness in FreeBSD 8.x ( possible leak/ threading issue )
Summary: Memory / syslog strangeness in FreeBSD 8.x ( possible leak/ threading issue )
Status: Open
Alias: None
Product: Base System
Classification: Unclassified
Component: threads (show other bugs)
Version: Unspecified
Hardware: Any Any
: Normal Affects Only Me
Assignee: freebsd-threads (Nobody)
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2010-07-12 09:10 UTC by Vikash
Modified: 2018-01-03 05:16 UTC (History)
0 users

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Vikash 2010-07-12 09:10:05 UTC
Can someone look into this strangeness on FreeBSD 8.0 and 8.1-RC2

using a threaded test code, I see the that freebsd 8.x seems to be using more memory than freebsd 7.x:


Results:

7.2
without syslog
  PID USERNAME    THR PRI NICE   SIZE    RES STATE    TIME   WCPU COMMAND
26872 vikashb    1001   8    0   128M 14236K RUN      0:00  0.00% a.out

with syslog
  PID USERNAME    THR PRI NICE   SIZE    RES STATE    TIME   WCPU COMMAND
26881 vikashb    1001  44    0   128M 26236K RUN      0:00  0.00% a.out

8.0-RELEASE-p3
without syslog
  PID USERNAME    THR PRI NICE   SIZE    RES STATE    TIME   WCPU COMMAND
61529 vikashb    1001  44    0   129M 14840K RUN      0:01  0.00% a.out

with syslog
  PID USERNAME    THR PRI NICE   SIZE    RES STATE    TIME   WCPU COMMAND
61507 vikashb    1001  44    0   257M 42708K RUN      0:30  0.00% a.out

8.1-RC2
without syslog
  PID USERNAME    THR PRI NICE   SIZE    RES STATE    TIME   WCPU COMMAND
33062 vikashb    1001  44    0   129M 14804K RUN      0:00  0.00% a.out

with syslog
  PID USERNAME    THR PRI NICE   SIZE    RES STATE    TIME   WCPU COMMAND
33056 vikashb    1001  44    0   257M 42708K RUN      0:03  0.00% a.out


I have not been able to find any reasonable information via Google/ PR search


either there is a leak or there is an undocumented behaviour in the 8.x threads.

neither freebsd-questions@freebsd.org nor freebsd-threads@freebsd.org yielded ant responses.

How-To-Repeat: compile the code below and observe the memory usage

8.x uses more memory than 7.x

<CODE>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/time.h>
#include <stdarg.h>
#include <errno.h>
#include <syslog.h>
#include <signal.h>
#include <pthread.h>

char *ProgramName = "WTF";

int loop = 0;
int LogToSTDOUT = 1;

void LogMessage(int debug, const char *fmt,...) {
   extern int LogToSTDOUT;

   char message[8192];

   memset(message, 0, sizeof(message));

   va_list args;

   va_start(args, fmt);
   vsnprintf(message, sizeof(message), fmt, args);
   va_end(args);

   if ( LogToSTDOUT )
   {
      printf("%s\n", message);
   }

   syslog(LOG_NOTICE, "%s", message);

}

unsigned long int getTimeNow()
{
   struct timeval tv;

   if ( gettimeofday(&tv, NULL) == -1 )
   {
      LogMessage(0, "ERROR(%d) %s\n", errno, strerror(errno));
      tv.tv_sec = 0;
   }

   return tv.tv_sec;
}

void HandleSignal(int sig)
{
   loop = 0;
   LogMessage(0, "loop  = %d\n", loop);
   signal(sig, SIG_IGN);
   usleep(1000);
}

void *worker(int n)
{
   while ( loop )
   {
      LogMessage(0, "worker #%d logging", n);
      usleep(1000);
   }

   pthread_exit(0);
}

int main(int argc, char* argv[])
{
   pthread_t* tpool;

   int workers = 1000, i, rc;

   openlog(ProgramName, LOG_PID, LOG_MAIL);

   unsigned long int duration = 120, StartTime, TimeNow;

   signal(SIGINT, HandleSignal);
   signal(SIGTERM, HandleSignal);
   signal(SIGHUP, HandleSignal);
   signal(SIGQUIT, HandleSignal);

   StartTime = getTimeNow();

   tpool = (pthread_t*)malloc(workers * sizeof(pthread_t));

   if ( tpool == NULL )
   {
      LogMessage(0, "malloc failed \n");
      closelog();
      exit(-1);
   }

   memset(tpool, 0, sizeof(pthread_t) * workers);

   loop = 1;

   for ( i = 0; i < workers; i++ )
   {
      rc = pthread_create(&tpool[i], NULL, (void *(*)(void*))&worker, (void*)i);

      if ( rc != 0 )
      {
         LogMessage(0, "pthread_create #%d failed\n", i );
         pthread_cancel(tpool[i]);
      }
      else
      {
         pthread_detach(tpool[i]);
      }
   }

   LogMessage(0, "loop  = %d\n", loop);

   while ( loop )
   {
      TimeNow = getTimeNow();

      if ( ( TimeNow - StartTime )  > duration )
      {
          loop = 0;
      }

      usleep(1000);
   }

   for ( i = 0; i < workers; i++ )
   {
      pthread_cancel(tpool[i]);
   }

   for ( i = 0; i < workers; i++ )
   {
      pthread_join(tpool[i], NULL);
   }

   free(tpool);
   closelog();
   exit(0);
}

</CODE>
Comment 1 Jilles Tjoelker freebsd_committer 2010-07-29 22:27:33 UTC
The difference could perhaps be explained by malloc(3) changes.
8.x's malloc tries to keep more memory per-thread, which reduces
contention but increases memory usage.

By the way, your program may behave erratically when it terminates or
when pthread_create() fails because pthread_cancel() may be called on an
uninitialized pthread_t, pthread functions are called on detached and
terminated threads and the signal handler calls functions that are not
async-signal-safe.

-- 
Jilles Tjoelker
Comment 2 Eitan Adler freebsd_committer freebsd_triage 2017-12-31 07:59:18 UTC
For bugs matching the following criteria:

Status: In Progress Changed: (is less than) 2014-06-01

Reset to default assignee and clear in-progress tags.

Mail being skipped