pax_global_header 0000666 0000000 0000000 00000000064 13041406524 0014511 g ustar 00root root 0000000 0000000 52 comment=2f153ea92a521bf7d2eb6b3108f393caafab3809
librdkafka-0.9.3/ 0000775 0000000 0000000 00000000000 13041406524 0013614 5 ustar 00root root 0000000 0000000 librdkafka-0.9.3/.dir-locals.el 0000664 0000000 0000000 00000000052 13041406524 0016242 0 ustar 00root root 0000000 0000000 ( (c-mode . ((c-file-style . "linux"))) )
librdkafka-0.9.3/.github/ 0000775 0000000 0000000 00000000000 13041406524 0015154 5 ustar 00root root 0000000 0000000 librdkafka-0.9.3/.github/ISSUE_TEMPLATE 0000664 0000000 0000000 00000000737 13041406524 0017271 0 ustar 00root root 0000000 0000000 Description
===========
How to reproduce
================
Checklist
=========
Please provide the following information:
- [ ] librdkafka version (release number or git tag):
- [ ] Apache Kafka version:
- [ ] librdkafka client configuration:
- [ ] Operating system:
- [ ] Using the legacy Consumer
- [ ] Using the high-level KafkaConsumer
- [ ] Provide logs (with `debug=..` as necessary) from librdkafka
- [ ] Provide broker log excerpts
- [ ] Critical issue
librdkafka-0.9.3/.gitignore 0000664 0000000 0000000 00000000322 13041406524 0015601 0 ustar 00root root 0000000 0000000 config.h
config.log*
config.cache
Makefile.config
rdkafka*.pc
*~
\#*
*.o
*.so
*.so.?
*.dylib
*.a
*.d
librdkafka*.lds
core
vgcore.*
*dSYM/
*.offset
SOURCES
gmon.out
*.zip
*.gz
*.bz2
*.deb
*.rpm
staging-docs
tmp
librdkafka-0.9.3/.travis.yml 0000664 0000000 0000000 00000000565 13041406524 0015733 0 ustar 00root root 0000000 0000000 language: c
cache: ccache
compiler:
- gcc
- clang
os:
- linux
- osx
before_script: ccache -s || echo "CCache is not available."
script:
- if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then CPPFLAGS="-I/usr/local/opt/openssl/include -L/usr/local/opt/openssl/lib" ./configure ; else ./configure ; fi
- make -j2 all examples check && make -C tests run_local
sudo: false
librdkafka-0.9.3/CONFIGURATION.md 0000664 0000000 0000000 00000056714 13041406524 0016162 0 ustar 00root root 0000000 0000000 //@file
## Global configuration properties
Property | C/P | Range | Default | Description
-----------------------------------------|-----|-----------------|--------------:|--------------------------
builtin.features | * | | gzip, snappy, ssl, sasl, regex, lz4 | Indicates the builtin features for this build of librdkafka. An application can either query this value or attempt to set it with its list of required features to check for library support. -.c file and
try to reproduce the issue in its most simple form.
Verify that the test case fails for earlier versions and passes with your
bugfix in-place.
New features and APIs should also result in an added test case.
Submitted patches must pass all existing tests.
For more information on the test suite see [tests/README]
## How to get your changes into the main sources
File a [pull request on github](https://github.com/edenhill/librdkafka/pulls)
Your change will be reviewed and discussed there and you will be
expected to correct flaws pointed out and update accordingly, or the change
risk stalling and eventually just get deleted without action. As a submitter
of a change, you are the owner of that change until it has been merged.
Make sure to monitor your PR on github and answer questions and/or
fix nits/flaws. This is very important. We will take lack of replies as a
sign that you're not very anxious to get your patch accepted and we tend to
simply drop such changes.
When you adjust your pull requests after review, please squash the
commits so that we can review the full updated version more easily
and keep history cleaner.
For example:
# Interactive rebase to let you squash/fixup commits
$ git rebase -i master
# Mark fixes-on-fixes commits as 'fixup' (or just 'f') in the
# first column. These will be silently integrated into the
# previous commit, so make sure to move the fixup-commit to
# the line beneath the parent commit.
# Since this probably rewrote the history of previously pushed
# commits you will need to make a force push, which is usually
# a bad idea but works good for pull requests.
$ git push --force origin your_feature_branch
### Write good commit messages
A short guide to how to write commit messages in the curl project.
---- start ----
[area]: [short line describing the main effect] [(#issuenumber)]
-- empty line --
[full description, no wider than 72 columns that describe as much as
possible as to why this change is made, and possibly what things
it fixes and everything else that is related]
---- stop ----
Example:
cgrp: restart query timer on all heartbeat failures (#10023)
If unhandled errors were received in HeartbeatResponse
the cgrp could get stuck in a state where it would not
refresh its coordinator.
# librdkafka C style guide
## Function and globals naming
Use self-explanatory hierarchical snake-case naming.
Pretty much all symbols should start with `rd_kafka_`, followed by
their subsystem (e.g., `cgrp`, `broker`, `buf`, etc..), followed by an
action (e.g, `find`, `get`, `clear`, ..).
## Variable naming
For existing types use the type prefix as variable name.
The type prefix is typically the first part of struct member fields.
Example:
* `rd_kafka_broker_t` has field names starting with `rkb_..`, thus broker
variable names should be named `rkb`
For other types use reasonably concise but descriptive names.
`i` and `j` are typical int iterators.
## Variable declaration
Variables must be declared at the head of a scope, no in-line variable
declarations are allowed.
## Indenting
Use 8 spaces indent, same as the Linux kernel.
In emacs, use `c-set-style "linux`.
For C++, use Google's C++ style.
## Comments
Use `/* .. */` comments, not `// ..`
For functions, use doxygen syntax, e.g.:
/**
* @brief
* ..
* @returns
*/
Make sure to comment non-obvious code and situations where the full
context of an operation is not easily graspable.
Also make sure to update existing comments when the code changes.
## Line length
Try hard to keep line length below 80 characters, when this is not possible
exceed it with reason.
## Braces
Braces go on the same line as their enveloping statement:
int some_func (..) {
while (1) {
if (1) {
do something;
..
} else {
do something else;
..
}
}
/* Single line scopes should not have braces */
if (1)
hi();
else if (2)
/* Say hello */
hello();
else
bye();
## Spaces
All expression parentheses should be prefixed and suffixed with a single space:
int some_func (int a) {
if (1)
....;
for (i = 0 ; i < 19 ; i++) {
}
}
Use space around operators:
int a = 2;
if (b >= 3)
c += 2;
Except for these:
d++;
--e;
## New block on new line
New blocks should be on a new line:
if (1)
new();
else
old();
## Parentheses
Don't assume the reader knows C operator precedence by heart for complex
statements, add parentheses to ease readability.
## ifdef hell
Avoid ifdef's as much as possible.
Platform support checking should be performed in configure.librdkafka.
# librdkafka C++ style guide
Follow [Google's C++ style guide](https://google.github.io/styleguide/cppguide.html)
librdkafka-0.9.3/Doxyfile 0000664 0000000 0000000 00000313433 13041406524 0015331 0 ustar 00root root 0000000 0000000 # Doxyfile 1.8.9.1
# This file describes the settings to be used by the documentation system
# doxygen (www.doxygen.org) for a project.
#
# All text after a double hash (##) is considered a comment and is placed in
# front of the TAG it is preceding.
#
# All text after a single hash (#) is considered a comment and will be ignored.
# The format is:
# TAG = value [value, ...]
# For lists, items can also be appended using:
# TAG += value [value, ...]
# Values that contain spaces should be placed between quotes (\" \").
#---------------------------------------------------------------------------
# Project related configuration options
#---------------------------------------------------------------------------
# This tag specifies the encoding used for all characters in the config file
# that follow. The default is UTF-8 which is also the encoding used for all text
# before the first occurrence of this tag. Doxygen uses libiconv (or the iconv
# built into libc) for the transcoding. See http://www.gnu.org/software/libiconv
# for the list of possible encodings.
# The default value is: UTF-8.
DOXYFILE_ENCODING = UTF-8
# The PROJECT_NAME tag is a single word (or a sequence of words surrounded by
# double-quotes, unless you are using Doxywizard) that should identify the
# project for which the documentation is generated. This name is used in the
# title of most generated pages and in a few other places.
# The default value is: My Project.
PROJECT_NAME = "librdkafka"
# The PROJECT_NUMBER tag can be used to enter a project or revision number. This
# could be handy for archiving the generated documentation or if some version
# control system is used.
PROJECT_NUMBER =
# Using the PROJECT_BRIEF tag one can provide an optional one line description
# for a project that appears at the top of each page and should give viewer a
# quick idea about the purpose of the project. Keep the description short.
PROJECT_BRIEF = "The Apache Kafka C/C++ client library"
# With the PROJECT_LOGO tag one can specify a logo or an icon that is included
# in the documentation. The maximum height of the logo should not exceed 55
# pixels and the maximum width should not exceed 200 pixels. Doxygen will copy
# the logo to the output directory.
#PROJECT_LOGO = kafka_logo.png
# The OUTPUT_DIRECTORY tag is used to specify the (relative or absolute) path
# into which the generated documentation will be written. If a relative path is
# entered, it will be relative to the location where doxygen was started. If
# left blank the current directory will be used.
OUTPUT_DIRECTORY = staging-docs
# If the CREATE_SUBDIRS tag is set to YES then doxygen will create 4096 sub-
# directories (in 2 levels) under the output directory of each output format and
# will distribute the generated files over these directories. Enabling this
# option can be useful when feeding doxygen a huge amount of source files, where
# putting all generated files in the same directory would otherwise causes
# performance problems for the file system.
# The default value is: NO.
CREATE_SUBDIRS = NO
# If the ALLOW_UNICODE_NAMES tag is set to YES, doxygen will allow non-ASCII
# characters to appear in the names of generated files. If set to NO, non-ASCII
# characters will be escaped, for example _xE3_x81_x84 will be used for Unicode
# U+3044.
# The default value is: NO.
ALLOW_UNICODE_NAMES = NO
# The OUTPUT_LANGUAGE tag is used to specify the language in which all
# documentation generated by doxygen is written. Doxygen will use this
# information to generate all constant output in the proper language.
# Possible values are: Afrikaans, Arabic, Armenian, Brazilian, Catalan, Chinese,
# Chinese-Traditional, Croatian, Czech, Danish, Dutch, English (United States),
# Esperanto, Farsi (Persian), Finnish, French, German, Greek, Hungarian,
# Indonesian, Italian, Japanese, Japanese-en (Japanese with English messages),
# Korean, Korean-en (Korean with English messages), Latvian, Lithuanian,
# Macedonian, Norwegian, Persian (Farsi), Polish, Portuguese, Romanian, Russian,
# Serbian, Serbian-Cyrillic, Slovak, Slovene, Spanish, Swedish, Turkish,
# Ukrainian and Vietnamese.
# The default value is: English.
OUTPUT_LANGUAGE = English
# If the BRIEF_MEMBER_DESC tag is set to YES, doxygen will include brief member
# descriptions after the members that are listed in the file and class
# documentation (similar to Javadoc). Set to NO to disable this.
# The default value is: YES.
BRIEF_MEMBER_DESC = YES
# If the REPEAT_BRIEF tag is set to YES, doxygen will prepend the brief
# description of a member or function before the detailed description
#
# Note: If both HIDE_UNDOC_MEMBERS and BRIEF_MEMBER_DESC are set to NO, the
# brief descriptions will be completely suppressed.
# The default value is: YES.
REPEAT_BRIEF = YES
# This tag implements a quasi-intelligent brief description abbreviator that is
# used to form the text in various listings. Each string in this list, if found
# as the leading text of the brief description, will be stripped from the text
# and the result, after processing the whole list, is used as the annotated
# text. Otherwise, the brief description is used as-is. If left blank, the
# following values are used ($name is automatically replaced with the name of
# the entity):The $name class, The $name widget, The $name file, is, provides,
# specifies, contains, represents, a, an and the.
ABBREVIATE_BRIEF =
# If the ALWAYS_DETAILED_SEC and REPEAT_BRIEF tags are both set to YES then
# doxygen will generate a detailed section even if there is only a brief
# description.
# The default value is: NO.
ALWAYS_DETAILED_SEC = NO
# If the INLINE_INHERITED_MEMB tag is set to YES, doxygen will show all
# inherited members of a class in the documentation of that class as if those
# members were ordinary class members. Constructors, destructors and assignment
# operators of the base classes will not be shown.
# The default value is: NO.
INLINE_INHERITED_MEMB = NO
# If the FULL_PATH_NAMES tag is set to YES, doxygen will prepend the full path
# before files name in the file list and in the header files. If set to NO the
# shortest path that makes the file name unique will be used
# The default value is: YES.
FULL_PATH_NAMES = YES
# The STRIP_FROM_PATH tag can be used to strip a user-defined part of the path.
# Stripping is only done if one of the specified strings matches the left-hand
# part of the path. The tag can be used to show relative paths in the file list.
# If left blank the directory from which doxygen is run is used as the path to
# strip.
#
# Note that you can specify absolute paths here, but also relative paths, which
# will be relative from the directory where doxygen is started.
# This tag requires that the tag FULL_PATH_NAMES is set to YES.
STRIP_FROM_PATH =
# The STRIP_FROM_INC_PATH tag can be used to strip a user-defined part of the
# path mentioned in the documentation of a class, which tells the reader which
# header file to include in order to use a class. If left blank only the name of
# the header file containing the class definition is used. Otherwise one should
# specify the list of include paths that are normally passed to the compiler
# using the -I flag.
STRIP_FROM_INC_PATH =
# If the SHORT_NAMES tag is set to YES, doxygen will generate much shorter (but
# less readable) file names. This can be useful is your file systems doesn't
# support long names like on DOS, Mac, or CD-ROM.
# The default value is: NO.
SHORT_NAMES = NO
# If the JAVADOC_AUTOBRIEF tag is set to YES then doxygen will interpret the
# first line (until the first dot) of a Javadoc-style comment as the brief
# description. If set to NO, the Javadoc-style will behave just like regular Qt-
# style comments (thus requiring an explicit @brief command for a brief
# description.)
# The default value is: NO.
JAVADOC_AUTOBRIEF = NO
# If the QT_AUTOBRIEF tag is set to YES then doxygen will interpret the first
# line (until the first dot) of a Qt-style comment as the brief description. If
# set to NO, the Qt-style will behave just like regular Qt-style comments (thus
# requiring an explicit \brief command for a brief description.)
# The default value is: NO.
QT_AUTOBRIEF = NO
# The MULTILINE_CPP_IS_BRIEF tag can be set to YES to make doxygen treat a
# multi-line C++ special comment block (i.e. a block of //! or /// comments) as
# a brief description. This used to be the default behavior. The new default is
# to treat a multi-line C++ comment block as a detailed description. Set this
# tag to YES if you prefer the old behavior instead.
#
# Note that setting this tag to YES also means that rational rose comments are
# not recognized any more.
# The default value is: NO.
MULTILINE_CPP_IS_BRIEF = NO
# If the INHERIT_DOCS tag is set to YES then an undocumented member inherits the
# documentation from any documented member that it re-implements.
# The default value is: YES.
INHERIT_DOCS = YES
# If the SEPARATE_MEMBER_PAGES tag is set to YES then doxygen will produce a new
# page for each member. If set to NO, the documentation of a member will be part
# of the file/class/namespace that contains it.
# The default value is: NO.
SEPARATE_MEMBER_PAGES = NO
# The TAB_SIZE tag can be used to set the number of spaces in a tab. Doxygen
# uses this value to replace tabs by spaces in code fragments.
# Minimum value: 1, maximum value: 16, default value: 4.
TAB_SIZE = 4
# This tag can be used to specify a number of aliases that act as commands in
# the documentation. An alias has the form:
# name=value
# For example adding
# "sideeffect=@par Side Effects:\n"
# will allow you to put the command \sideeffect (or @sideeffect) in the
# documentation, which will result in a user-defined paragraph with heading
# "Side Effects:". You can put \n's in the value part of an alias to insert
# newlines.
ALIASES = "locality=@par Thread restriction:"
# This tag can be used to specify a number of word-keyword mappings (TCL only).
# A mapping has the form "name=value". For example adding "class=itcl::class"
# will allow you to use the command class in the itcl::class meaning.
TCL_SUBST =
# Set the OPTIMIZE_OUTPUT_FOR_C tag to YES if your project consists of C sources
# only. Doxygen will then generate output that is more tailored for C. For
# instance, some of the names that are used will be different. The list of all
# members will be omitted, etc.
# The default value is: NO.
OPTIMIZE_OUTPUT_FOR_C = YES
# Set the OPTIMIZE_OUTPUT_JAVA tag to YES if your project consists of Java or
# Python sources only. Doxygen will then generate output that is more tailored
# for that language. For instance, namespaces will be presented as packages,
# qualified scopes will look different, etc.
# The default value is: NO.
OPTIMIZE_OUTPUT_JAVA = NO
# Set the OPTIMIZE_FOR_FORTRAN tag to YES if your project consists of Fortran
# sources. Doxygen will then generate output that is tailored for Fortran.
# The default value is: NO.
OPTIMIZE_FOR_FORTRAN = NO
# Set the OPTIMIZE_OUTPUT_VHDL tag to YES if your project consists of VHDL
# sources. Doxygen will then generate output that is tailored for VHDL.
# The default value is: NO.
OPTIMIZE_OUTPUT_VHDL = NO
# Doxygen selects the parser to use depending on the extension of the files it
# parses. With this tag you can assign which parser to use for a given
# extension. Doxygen has a built-in mapping, but you can override or extend it
# using this tag. The format is ext=language, where ext is a file extension, and
# language is one of the parsers supported by doxygen: IDL, Java, Javascript,
# C#, C, C++, D, PHP, Objective-C, Python, Fortran (fixed format Fortran:
# FortranFixed, free formatted Fortran: FortranFree, unknown formatted Fortran:
# Fortran. In the later case the parser tries to guess whether the code is fixed
# or free formatted code, this is the default for Fortran type files), VHDL. For
# instance to make doxygen treat .inc files as Fortran files (default is PHP),
# and .f files as C (default is Fortran), use: inc=Fortran f=C.
#
# Note: For files without extension you can use no_extension as a placeholder.
#
# Note that for custom extensions you also need to set FILE_PATTERNS otherwise
# the files are not read by doxygen.
EXTENSION_MAPPING =
# If the MARKDOWN_SUPPORT tag is enabled then doxygen pre-processes all comments
# according to the Markdown format, which allows for more readable
# documentation. See http://daringfireball.net/projects/markdown/ for details.
# The output of markdown processing is further processed by doxygen, so you can
# mix doxygen, HTML, and XML commands with Markdown formatting. Disable only in
# case of backward compatibilities issues.
# The default value is: YES.
MARKDOWN_SUPPORT = YES
# When enabled doxygen tries to link words that correspond to documented
# classes, or namespaces to their corresponding documentation. Such a link can
# be prevented in individual cases by putting a % sign in front of the word or
# globally by setting AUTOLINK_SUPPORT to NO.
# The default value is: YES.
AUTOLINK_SUPPORT = YES
# If you use STL classes (i.e. std::string, std::vector, etc.) but do not want
# to include (a tag file for) the STL sources as input, then you should set this
# tag to YES in order to let doxygen match functions declarations and
# definitions whose arguments contain STL classes (e.g. func(std::string);
# versus func(std::string) {}). This also make the inheritance and collaboration
# diagrams that involve STL classes more complete and accurate.
# The default value is: NO.
BUILTIN_STL_SUPPORT = NO
# If you use Microsoft's C++/CLI language, you should set this option to YES to
# enable parsing support.
# The default value is: NO.
CPP_CLI_SUPPORT = NO
# Set the SIP_SUPPORT tag to YES if your project consists of sip (see:
# http://www.riverbankcomputing.co.uk/software/sip/intro) sources only. Doxygen
# will parse them like normal C++ but will assume all classes use public instead
# of private inheritance when no explicit protection keyword is present.
# The default value is: NO.
SIP_SUPPORT = NO
# For Microsoft's IDL there are propget and propput attributes to indicate
# getter and setter methods for a property. Setting this option to YES will make
# doxygen to replace the get and set methods by a property in the documentation.
# This will only work if the methods are indeed getting or setting a simple
# type. If this is not the case, or you want to show the methods anyway, you
# should set this option to NO.
# The default value is: YES.
IDL_PROPERTY_SUPPORT = YES
# If member grouping is used in the documentation and the DISTRIBUTE_GROUP_DOC
# tag is set to YES then doxygen will reuse the documentation of the first
# member in the group (if any) for the other members of the group. By default
# all members of a group must be documented explicitly.
# The default value is: NO.
DISTRIBUTE_GROUP_DOC = NO
# Set the SUBGROUPING tag to YES to allow class member groups of the same type
# (for instance a group of public functions) to be put as a subgroup of that
# type (e.g. under the Public Functions section). Set it to NO to prevent
# subgrouping. Alternatively, this can be done per class using the
# \nosubgrouping command.
# The default value is: YES.
SUBGROUPING = YES
# When the INLINE_GROUPED_CLASSES tag is set to YES, classes, structs and unions
# are shown inside the group in which they are included (e.g. using \ingroup)
# instead of on a separate page (for HTML and Man pages) or section (for LaTeX
# and RTF).
#
# Note that this feature does not work in combination with
# SEPARATE_MEMBER_PAGES.
# The default value is: NO.
INLINE_GROUPED_CLASSES = NO
# When the INLINE_SIMPLE_STRUCTS tag is set to YES, structs, classes, and unions
# with only public data fields or simple typedef fields will be shown inline in
# the documentation of the scope in which they are defined (i.e. file,
# namespace, or group documentation), provided this scope is documented. If set
# to NO, structs, classes, and unions are shown on a separate page (for HTML and
# Man pages) or section (for LaTeX and RTF).
# The default value is: NO.
INLINE_SIMPLE_STRUCTS = NO
# When TYPEDEF_HIDES_STRUCT tag is enabled, a typedef of a struct, union, or
# enum is documented as struct, union, or enum with the name of the typedef. So
# typedef struct TypeS {} TypeT, will appear in the documentation as a struct
# with name TypeT. When disabled the typedef will appear as a member of a file,
# namespace, or class. And the struct will be named TypeS. This can typically be
# useful for C code in case the coding convention dictates that all compound
# types are typedef'ed and only the typedef is referenced, never the tag name.
# The default value is: NO.
TYPEDEF_HIDES_STRUCT = YES
# The size of the symbol lookup cache can be set using LOOKUP_CACHE_SIZE. This
# cache is used to resolve symbols given their name and scope. Since this can be
# an expensive process and often the same symbol appears multiple times in the
# code, doxygen keeps a cache of pre-resolved symbols. If the cache is too small
# doxygen will become slower. If the cache is too large, memory is wasted. The
# cache size is given by this formula: 2^(16+LOOKUP_CACHE_SIZE). The valid range
# is 0..9, the default is 0, corresponding to a cache size of 2^16=65536
# symbols. At the end of a run doxygen will report the cache usage and suggest
# the optimal cache size from a speed point of view.
# Minimum value: 0, maximum value: 9, default value: 0.
LOOKUP_CACHE_SIZE = 0
#---------------------------------------------------------------------------
# Build related configuration options
#---------------------------------------------------------------------------
# If the EXTRACT_ALL tag is set to YES, doxygen will assume all entities in
# documentation are documented, even if no documentation was available. Private
# class members and static file members will be hidden unless the
# EXTRACT_PRIVATE respectively EXTRACT_STATIC tags are set to YES.
# Note: This will also disable the warnings about undocumented members that are
# normally produced when WARNINGS is set to YES.
# The default value is: NO.
EXTRACT_ALL = NO
# If the EXTRACT_PRIVATE tag is set to YES, all private members of a class will
# be included in the documentation.
# The default value is: NO.
EXTRACT_PRIVATE = NO
# If the EXTRACT_PACKAGE tag is set to YES, all members with package or internal
# scope will be included in the documentation.
# The default value is: NO.
EXTRACT_PACKAGE = NO
# If the EXTRACT_STATIC tag is set to YES, all static members of a file will be
# included in the documentation.
# The default value is: NO.
EXTRACT_STATIC = YES
# If the EXTRACT_LOCAL_CLASSES tag is set to YES, classes (and structs) defined
# locally in source files will be included in the documentation. If set to NO,
# only classes defined in header files are included. Does not have any effect
# for Java sources.
# The default value is: YES.
EXTRACT_LOCAL_CLASSES = YES
# This flag is only useful for Objective-C code. If set to YES, local methods,
# which are defined in the implementation section but not in the interface are
# included in the documentation. If set to NO, only methods in the interface are
# included.
# The default value is: NO.
EXTRACT_LOCAL_METHODS = NO
# If this flag is set to YES, the members of anonymous namespaces will be
# extracted and appear in the documentation as a namespace called
# 'anonymous_namespace{file}', where file will be replaced with the base name of
# the file that contains the anonymous namespace. By default anonymous namespace
# are hidden.
# The default value is: NO.
EXTRACT_ANON_NSPACES = NO
# If the HIDE_UNDOC_MEMBERS tag is set to YES, doxygen will hide all
# undocumented members inside documented classes or files. If set to NO these
# members will be included in the various overviews, but no documentation
# section is generated. This option has no effect if EXTRACT_ALL is enabled.
# The default value is: NO.
HIDE_UNDOC_MEMBERS = NO
# If the HIDE_UNDOC_CLASSES tag is set to YES, doxygen will hide all
# undocumented classes that are normally visible in the class hierarchy. If set
# to NO, these classes will be included in the various overviews. This option
# has no effect if EXTRACT_ALL is enabled.
# The default value is: NO.
HIDE_UNDOC_CLASSES = NO
# If the HIDE_FRIEND_COMPOUNDS tag is set to YES, doxygen will hide all friend
# (class|struct|union) declarations. If set to NO, these declarations will be
# included in the documentation.
# The default value is: NO.
HIDE_FRIEND_COMPOUNDS = NO
# If the HIDE_IN_BODY_DOCS tag is set to YES, doxygen will hide any
# documentation blocks found inside the body of a function. If set to NO, these
# blocks will be appended to the function's detailed documentation block.
# The default value is: NO.
HIDE_IN_BODY_DOCS = NO
# The INTERNAL_DOCS tag determines if documentation that is typed after a
# \internal command is included. If the tag is set to NO then the documentation
# will be excluded. Set it to YES to include the internal documentation.
# The default value is: NO.
INTERNAL_DOCS = NO
# If the CASE_SENSE_NAMES tag is set to NO then doxygen will only generate file
# names in lower-case letters. If set to YES, upper-case letters are also
# allowed. This is useful if you have classes or files whose names only differ
# in case and if your file system supports case sensitive file names. Windows
# and Mac users are advised to set this option to NO.
# The default value is: system dependent.
CASE_SENSE_NAMES = YES
# If the HIDE_SCOPE_NAMES tag is set to NO then doxygen will show members with
# their full class and namespace scopes in the documentation. If set to YES, the
# scope will be hidden.
# The default value is: NO.
HIDE_SCOPE_NAMES = NO
# If the HIDE_COMPOUND_REFERENCE tag is set to NO (default) then doxygen will
# append additional text to a page's title, such as Class Reference. If set to
# YES the compound reference will be hidden.
# The default value is: NO.
HIDE_COMPOUND_REFERENCE= NO
# If the SHOW_INCLUDE_FILES tag is set to YES then doxygen will put a list of
# the files that are included by a file in the documentation of that file.
# The default value is: YES.
SHOW_INCLUDE_FILES = YES
# If the SHOW_GROUPED_MEMB_INC tag is set to YES then Doxygen will add for each
# grouped member an include statement to the documentation, telling the reader
# which file to include in order to use the member.
# The default value is: NO.
SHOW_GROUPED_MEMB_INC = NO
# If the FORCE_LOCAL_INCLUDES tag is set to YES then doxygen will list include
# files with double quotes in the documentation rather than with sharp brackets.
# The default value is: NO.
FORCE_LOCAL_INCLUDES = NO
# If the INLINE_INFO tag is set to YES then a tag [inline] is inserted in the
# documentation for inline members.
# The default value is: YES.
INLINE_INFO = YES
# If the SORT_MEMBER_DOCS tag is set to YES then doxygen will sort the
# (detailed) documentation of file and class members alphabetically by member
# name. If set to NO, the members will appear in declaration order.
# The default value is: YES.
SORT_MEMBER_DOCS = NO
# If the SORT_BRIEF_DOCS tag is set to YES then doxygen will sort the brief
# descriptions of file, namespace and class members alphabetically by member
# name. If set to NO, the members will appear in declaration order. Note that
# this will also influence the order of the classes in the class list.
# The default value is: NO.
SORT_BRIEF_DOCS = NO
# If the SORT_MEMBERS_CTORS_1ST tag is set to YES then doxygen will sort the
# (brief and detailed) documentation of class members so that constructors and
# destructors are listed first. If set to NO the constructors will appear in the
# respective orders defined by SORT_BRIEF_DOCS and SORT_MEMBER_DOCS.
# Note: If SORT_BRIEF_DOCS is set to NO this option is ignored for sorting brief
# member documentation.
# Note: If SORT_MEMBER_DOCS is set to NO this option is ignored for sorting
# detailed member documentation.
# The default value is: NO.
SORT_MEMBERS_CTORS_1ST = NO
# If the SORT_GROUP_NAMES tag is set to YES then doxygen will sort the hierarchy
# of group names into alphabetical order. If set to NO the group names will
# appear in their defined order.
# The default value is: NO.
SORT_GROUP_NAMES = NO
# If the SORT_BY_SCOPE_NAME tag is set to YES, the class list will be sorted by
# fully-qualified names, including namespaces. If set to NO, the class list will
# be sorted only by class name, not including the namespace part.
# Note: This option is not very useful if HIDE_SCOPE_NAMES is set to YES.
# Note: This option applies only to the class list, not to the alphabetical
# list.
# The default value is: NO.
SORT_BY_SCOPE_NAME = NO
# If the STRICT_PROTO_MATCHING option is enabled and doxygen fails to do proper
# type resolution of all parameters of a function it will reject a match between
# the prototype and the implementation of a member function even if there is
# only one candidate or it is obvious which candidate to choose by doing a
# simple string match. By disabling STRICT_PROTO_MATCHING doxygen will still
# accept a match between prototype and implementation in such cases.
# The default value is: NO.
STRICT_PROTO_MATCHING = NO
# The GENERATE_TODOLIST tag can be used to enable (YES) or disable (NO) the todo
# list. This list is created by putting \todo commands in the documentation.
# The default value is: YES.
GENERATE_TODOLIST = YES
# The GENERATE_TESTLIST tag can be used to enable (YES) or disable (NO) the test
# list. This list is created by putting \test commands in the documentation.
# The default value is: YES.
GENERATE_TESTLIST = YES
# The GENERATE_BUGLIST tag can be used to enable (YES) or disable (NO) the bug
# list. This list is created by putting \bug commands in the documentation.
# The default value is: YES.
GENERATE_BUGLIST = YES
# The GENERATE_DEPRECATEDLIST tag can be used to enable (YES) or disable (NO)
# the deprecated list. This list is created by putting \deprecated commands in
# the documentation.
# The default value is: YES.
GENERATE_DEPRECATEDLIST= YES
# The ENABLED_SECTIONS tag can be used to enable conditional documentation
# sections, marked by \if ... \endif and \cond
# ... \endcond blocks.
ENABLED_SECTIONS =
# The MAX_INITIALIZER_LINES tag determines the maximum number of lines that the
# initial value of a variable or macro / define can have for it to appear in the
# documentation. If the initializer consists of more lines than specified here
# it will be hidden. Use a value of 0 to hide initializers completely. The
# appearance of the value of individual variables and macros / defines can be
# controlled using \showinitializer or \hideinitializer command in the
# documentation regardless of this setting.
# Minimum value: 0, maximum value: 10000, default value: 30.
MAX_INITIALIZER_LINES = 30
# Set the SHOW_USED_FILES tag to NO to disable the list of files generated at
# the bottom of the documentation of classes and structs. If set to YES, the
# list will mention the files that were used to generate the documentation.
# The default value is: YES.
SHOW_USED_FILES = YES
# Set the SHOW_FILES tag to NO to disable the generation of the Files page. This
# will remove the Files entry from the Quick Index and from the Folder Tree View
# (if specified).
# The default value is: YES.
SHOW_FILES = YES
# Set the SHOW_NAMESPACES tag to NO to disable the generation of the Namespaces
# page. This will remove the Namespaces entry from the Quick Index and from the
# Folder Tree View (if specified).
# The default value is: YES.
SHOW_NAMESPACES = NO
# The FILE_VERSION_FILTER tag can be used to specify a program or script that
# doxygen should invoke to get the current version for each file (typically from
# the version control system). Doxygen will invoke the program by executing (via
# popen()) the command command input-file, where command is the value of the
# FILE_VERSION_FILTER tag, and input-file is the name of an input file provided
# by doxygen. Whatever the program writes to standard output is used as the file
# version. For an example see the documentation.
FILE_VERSION_FILTER =
# The LAYOUT_FILE tag can be used to specify a layout file which will be parsed
# by doxygen. The layout file controls the global structure of the generated
# output files in an output format independent way. To create the layout file
# that represents doxygen's defaults, run doxygen with the -l option. You can
# optionally specify a file name after the option, if omitted DoxygenLayout.xml
# will be used as the name of the layout file.
#
# Note that if you run doxygen from a directory containing a file called
# DoxygenLayout.xml, doxygen will parse it automatically even if the LAYOUT_FILE
# tag is left empty.
LAYOUT_FILE =
# The CITE_BIB_FILES tag can be used to specify one or more bib files containing
# the reference definitions. This must be a list of .bib files. The .bib
# extension is automatically appended if omitted. This requires the bibtex tool
# to be installed. See also http://en.wikipedia.org/wiki/BibTeX for more info.
# For LaTeX the style of the bibliography can be controlled using
# LATEX_BIB_STYLE. To use this feature you need bibtex and perl available in the
# search path. See also \cite for info how to create references.
CITE_BIB_FILES =
#---------------------------------------------------------------------------
# Configuration options related to warning and progress messages
#---------------------------------------------------------------------------
# The QUIET tag can be used to turn on/off the messages that are generated to
# standard output by doxygen. If QUIET is set to YES this implies that the
# messages are off.
# The default value is: NO.
QUIET = NO
# The WARNINGS tag can be used to turn on/off the warning messages that are
# generated to standard error (stderr) by doxygen. If WARNINGS is set to YES
# this implies that the warnings are on.
#
# Tip: Turn warnings on while writing the documentation.
# The default value is: YES.
WARNINGS = YES
# If the WARN_IF_UNDOCUMENTED tag is set to YES then doxygen will generate
# warnings for undocumented members. If EXTRACT_ALL is set to YES then this flag
# will automatically be disabled.
# The default value is: YES.
WARN_IF_UNDOCUMENTED = YES
# If the WARN_IF_DOC_ERROR tag is set to YES, doxygen will generate warnings for
# potential errors in the documentation, such as not documenting some parameters
# in a documented function, or documenting parameters that don't exist or using
# markup commands wrongly.
# The default value is: YES.
WARN_IF_DOC_ERROR = YES
# This WARN_NO_PARAMDOC option can be enabled to get warnings for functions that
# are documented, but have no documentation for their parameters or return
# value. If set to NO, doxygen will only warn about wrong or incomplete
# parameter documentation, but not about the absence of documentation.
# The default value is: NO.
WARN_NO_PARAMDOC = NO
# The WARN_FORMAT tag determines the format of the warning messages that doxygen
# can produce. The string should contain the $file, $line, and $text tags, which
# will be replaced by the file and line number from which the warning originated
# and the warning text. Optionally the format may contain $version, which will
# be replaced by the version of the file (if it could be obtained via
# FILE_VERSION_FILTER)
# The default value is: $file:$line: $text.
WARN_FORMAT = "$file:$line: $text"
# The WARN_LOGFILE tag can be used to specify a file to which warning and error
# messages should be written. If left blank the output is written to standard
# error (stderr).
WARN_LOGFILE =
#---------------------------------------------------------------------------
# Configuration options related to the input files
#---------------------------------------------------------------------------
# The INPUT tag is used to specify the files and/or directories that contain
# documented source files. You may enter file names like myfile.cpp or
# directories like /usr/src/myproject. Separate the files or directories with
# spaces.
# Note: If this tag is empty the current directory is searched.
INPUT = mainpage.doxy INTRODUCTION.md CONFIGURATION.md src/rdkafka.h src-cpp/rdkafkacpp.h
# This tag can be used to specify the character encoding of the source files
# that doxygen parses. Internally doxygen uses the UTF-8 encoding. Doxygen uses
# libiconv (or the iconv built into libc) for the transcoding. See the libiconv
# documentation (see: http://www.gnu.org/software/libiconv) for the list of
# possible encodings.
# The default value is: UTF-8.
INPUT_ENCODING = UTF-8
# If the value of the INPUT tag contains directories, you can use the
# FILE_PATTERNS tag to specify one or more wildcard patterns (like *.cpp and
# *.h) to filter out the source-files in the directories. If left blank the
# following patterns are tested:*.c, *.cc, *.cxx, *.cpp, *.c++, *.java, *.ii,
# *.ixx, *.ipp, *.i++, *.inl, *.idl, *.ddl, *.odl, *.h, *.hh, *.hxx, *.hpp,
# *.h++, *.cs, *.d, *.php, *.php4, *.php5, *.phtml, *.inc, *.m, *.markdown,
# *.md, *.mm, *.dox, *.py, *.f90, *.f, *.for, *.tcl, *.vhd, *.vhdl, *.ucf,
# *.qsf, *.as and *.js.
FILE_PATTERNS =
# The RECURSIVE tag can be used to specify whether or not subdirectories should
# be searched for input files as well.
# The default value is: NO.
RECURSIVE = NO
# The EXCLUDE tag can be used to specify files and/or directories that should be
# excluded from the INPUT source files. This way you can easily exclude a
# subdirectory from a directory tree whose root is specified with the INPUT tag.
#
# Note that relative paths are relative to the directory from which doxygen is
# run.
EXCLUDE =
# The EXCLUDE_SYMLINKS tag can be used to select whether or not files or
# directories that are symbolic links (a Unix file system feature) are excluded
# from the input.
# The default value is: NO.
EXCLUDE_SYMLINKS = NO
# If the value of the INPUT tag contains directories, you can use the
# EXCLUDE_PATTERNS tag to specify one or more wildcard patterns to exclude
# certain files from those directories.
#
# Note that the wildcards are matched against the file with absolute path, so to
# exclude all test directories for example use the pattern */test/*
EXCLUDE_PATTERNS =
# The EXCLUDE_SYMBOLS tag can be used to specify one or more symbol names
# (namespaces, classes, functions, etc.) that should be excluded from the
# output. The symbol name can be a fully qualified name, a word, or if the
# wildcard * is used, a substring. Examples: ANamespace, AClass,
# AClass::ANamespace, ANamespace::*Test
#
# Note that the wildcards are matched against the file with absolute path, so to
# exclude all test directories use the pattern */test/*
EXCLUDE_SYMBOLS =
# The EXAMPLE_PATH tag can be used to specify one or more files or directories
# that contain example code fragments that are included (see the \include
# command).
EXAMPLE_PATH =
# If the value of the EXAMPLE_PATH tag contains directories, you can use the
# EXAMPLE_PATTERNS tag to specify one or more wildcard pattern (like *.cpp and
# *.h) to filter out the source-files in the directories. If left blank all
# files are included.
EXAMPLE_PATTERNS =
# If the EXAMPLE_RECURSIVE tag is set to YES then subdirectories will be
# searched for input files to be used with the \include or \dontinclude commands
# irrespective of the value of the RECURSIVE tag.
# The default value is: NO.
EXAMPLE_RECURSIVE = NO
# The IMAGE_PATH tag can be used to specify one or more files or directories
# that contain images that are to be included in the documentation (see the
# \image command).
IMAGE_PATH =
# The INPUT_FILTER tag can be used to specify a program that doxygen should
# invoke to filter for each input file. Doxygen will invoke the filter program
# by executing (via popen()) the command:
#
#
#
# where is the value of the INPUT_FILTER tag, and is the
# name of an input file. Doxygen will then use the output that the filter
# program writes to standard output. If FILTER_PATTERNS is specified, this tag
# will be ignored.
#
# Note that the filter must not add or remove lines; it is applied before the
# code is scanned, but not when the output code is generated. If lines are added
# or removed, the anchors will not be placed correctly.
INPUT_FILTER =
# The FILTER_PATTERNS tag can be used to specify filters on a per file pattern
# basis. Doxygen will compare the file name with each pattern and apply the
# filter if there is a match. The filters are a list of the form: pattern=filter
# (like *.cpp=my_cpp_filter). See INPUT_FILTER for further information on how
# filters are used. If the FILTER_PATTERNS tag is empty or if none of the
# patterns match the file name, INPUT_FILTER is applied.
FILTER_PATTERNS =
# If the FILTER_SOURCE_FILES tag is set to YES, the input filter (if set using
# INPUT_FILTER) will also be used to filter the input files that are used for
# producing the source files to browse (i.e. when SOURCE_BROWSER is set to YES).
# The default value is: NO.
FILTER_SOURCE_FILES = NO
# The FILTER_SOURCE_PATTERNS tag can be used to specify source filters per file
# pattern. A pattern will override the setting for FILTER_PATTERN (if any) and
# it is also possible to disable source filtering for a specific pattern using
# *.ext= (so without naming a filter).
# This tag requires that the tag FILTER_SOURCE_FILES is set to YES.
FILTER_SOURCE_PATTERNS =
# If the USE_MDFILE_AS_MAINPAGE tag refers to the name of a markdown file that
# is part of the input, its contents will be placed on the main page
# (index.html). This can be useful if you have a project on for instance GitHub
# and want to reuse the introduction page also for the doxygen output.
USE_MDFILE_AS_MAINPAGE =
#---------------------------------------------------------------------------
# Configuration options related to source browsing
#---------------------------------------------------------------------------
# If the SOURCE_BROWSER tag is set to YES then a list of source files will be
# generated. Documented entities will be cross-referenced with these sources.
#
# Note: To get rid of all source code in the generated output, make sure that
# also VERBATIM_HEADERS is set to NO.
# The default value is: NO.
SOURCE_BROWSER = NO
# Setting the INLINE_SOURCES tag to YES will include the body of functions,
# classes and enums directly into the documentation.
# The default value is: NO.
INLINE_SOURCES = NO
# Setting the STRIP_CODE_COMMENTS tag to YES will instruct doxygen to hide any
# special comment blocks from generated source code fragments. Normal C, C++ and
# Fortran comments will always remain visible.
# The default value is: YES.
STRIP_CODE_COMMENTS = YES
# If the REFERENCED_BY_RELATION tag is set to YES then for each documented
# function all documented functions referencing it will be listed.
# The default value is: NO.
REFERENCED_BY_RELATION = NO
# If the REFERENCES_RELATION tag is set to YES then for each documented function
# all documented entities called/used by that function will be listed.
# The default value is: NO.
REFERENCES_RELATION = NO
# If the REFERENCES_LINK_SOURCE tag is set to YES and SOURCE_BROWSER tag is set
# to YES then the hyperlinks from functions in REFERENCES_RELATION and
# REFERENCED_BY_RELATION lists will link to the source code. Otherwise they will
# link to the documentation.
# The default value is: YES.
REFERENCES_LINK_SOURCE = YES
# If SOURCE_TOOLTIPS is enabled (the default) then hovering a hyperlink in the
# source code will show a tooltip with additional information such as prototype,
# brief description and links to the definition and documentation. Since this
# will make the HTML file larger and loading of large files a bit slower, you
# can opt to disable this feature.
# The default value is: YES.
# This tag requires that the tag SOURCE_BROWSER is set to YES.
SOURCE_TOOLTIPS = YES
# If the USE_HTAGS tag is set to YES then the references to source code will
# point to the HTML generated by the htags(1) tool instead of doxygen built-in
# source browser. The htags tool is part of GNU's global source tagging system
# (see http://www.gnu.org/software/global/global.html). You will need version
# 4.8.6 or higher.
#
# To use it do the following:
# - Install the latest version of global
# - Enable SOURCE_BROWSER and USE_HTAGS in the config file
# - Make sure the INPUT points to the root of the source tree
# - Run doxygen as normal
#
# Doxygen will invoke htags (and that will in turn invoke gtags), so these
# tools must be available from the command line (i.e. in the search path).
#
# The result: instead of the source browser generated by doxygen, the links to
# source code will now point to the output of htags.
# The default value is: NO.
# This tag requires that the tag SOURCE_BROWSER is set to YES.
USE_HTAGS = NO
# If the VERBATIM_HEADERS tag is set the YES then doxygen will generate a
# verbatim copy of the header file for each class for which an include is
# specified. Set to NO to disable this.
# See also: Section \class.
# The default value is: YES.
VERBATIM_HEADERS = YES
# If the CLANG_ASSISTED_PARSING tag is set to YES then doxygen will use the
# clang parser (see: http://clang.llvm.org/) for more accurate parsing at the
# cost of reduced performance. This can be particularly helpful with template
# rich C++ code for which doxygen's built-in parser lacks the necessary type
# information.
# Note: The availability of this option depends on whether or not doxygen was
# compiled with the --with-libclang option.
# The default value is: NO.
CLANG_ASSISTED_PARSING = NO
# If clang assisted parsing is enabled you can provide the compiler with command
# line options that you would normally use when invoking the compiler. Note that
# the include paths will already be set by doxygen for the files and directories
# specified with INPUT and INCLUDE_PATH.
# This tag requires that the tag CLANG_ASSISTED_PARSING is set to YES.
CLANG_OPTIONS =
#---------------------------------------------------------------------------
# Configuration options related to the alphabetical class index
#---------------------------------------------------------------------------
# If the ALPHABETICAL_INDEX tag is set to YES, an alphabetical index of all
# compounds will be generated. Enable this if the project contains a lot of
# classes, structs, unions or interfaces.
# The default value is: YES.
ALPHABETICAL_INDEX = YES
# The COLS_IN_ALPHA_INDEX tag can be used to specify the number of columns in
# which the alphabetical index list will be split.
# Minimum value: 1, maximum value: 20, default value: 5.
# This tag requires that the tag ALPHABETICAL_INDEX is set to YES.
COLS_IN_ALPHA_INDEX = 5
# In case all classes in a project start with a common prefix, all classes will
# be put under the same header in the alphabetical index. The IGNORE_PREFIX tag
# can be used to specify a prefix (or a list of prefixes) that should be ignored
# while generating the index headers.
# This tag requires that the tag ALPHABETICAL_INDEX is set to YES.
IGNORE_PREFIX =
#---------------------------------------------------------------------------
# Configuration options related to the HTML output
#---------------------------------------------------------------------------
# If the GENERATE_HTML tag is set to YES, doxygen will generate HTML output
# The default value is: YES.
GENERATE_HTML = YES
# The HTML_OUTPUT tag is used to specify where the HTML docs will be put. If a
# relative path is entered the value of OUTPUT_DIRECTORY will be put in front of
# it.
# The default directory is: html.
# This tag requires that the tag GENERATE_HTML is set to YES.
HTML_OUTPUT = html
# The HTML_FILE_EXTENSION tag can be used to specify the file extension for each
# generated HTML page (for example: .htm, .php, .asp).
# The default value is: .html.
# This tag requires that the tag GENERATE_HTML is set to YES.
HTML_FILE_EXTENSION = .html
# The HTML_HEADER tag can be used to specify a user-defined HTML header file for
# each generated HTML page. If the tag is left blank doxygen will generate a
# standard header.
#
# To get valid HTML the header file that includes any scripts and style sheets
# that doxygen needs, which is dependent on the configuration options used (e.g.
# the setting GENERATE_TREEVIEW). It is highly recommended to start with a
# default header using
# doxygen -w html new_header.html new_footer.html new_stylesheet.css
# YourConfigFile
# and then modify the file new_header.html. See also section "Doxygen usage"
# for information on how to generate the default header that doxygen normally
# uses.
# Note: The header is subject to change so you typically have to regenerate the
# default header when upgrading to a newer version of doxygen. For a description
# of the possible markers and block names see the documentation.
# This tag requires that the tag GENERATE_HTML is set to YES.
HTML_HEADER =
# The HTML_FOOTER tag can be used to specify a user-defined HTML footer for each
# generated HTML page. If the tag is left blank doxygen will generate a standard
# footer. See HTML_HEADER for more information on how to generate a default
# footer and what special commands can be used inside the footer. See also
# section "Doxygen usage" for information on how to generate the default footer
# that doxygen normally uses.
# This tag requires that the tag GENERATE_HTML is set to YES.
HTML_FOOTER =
# The HTML_STYLESHEET tag can be used to specify a user-defined cascading style
# sheet that is used by each HTML page. It can be used to fine-tune the look of
# the HTML output. If left blank doxygen will generate a default style sheet.
# See also section "Doxygen usage" for information on how to generate the style
# sheet that doxygen normally uses.
# Note: It is recommended to use HTML_EXTRA_STYLESHEET instead of this tag, as
# it is more robust and this tag (HTML_STYLESHEET) will in the future become
# obsolete.
# This tag requires that the tag GENERATE_HTML is set to YES.
HTML_STYLESHEET =
# The HTML_EXTRA_STYLESHEET tag can be used to specify additional user-defined
# cascading style sheets that are included after the standard style sheets
# created by doxygen. Using this option one can overrule certain style aspects.
# This is preferred over using HTML_STYLESHEET since it does not replace the
# standard style sheet and is therefore more robust against future updates.
# Doxygen will copy the style sheet files to the output directory.
# Note: The order of the extra style sheet files is of importance (e.g. the last
# style sheet in the list overrules the setting of the previous ones in the
# list). For an example see the documentation.
# This tag requires that the tag GENERATE_HTML is set to YES.
HTML_EXTRA_STYLESHEET =
# The HTML_EXTRA_FILES tag can be used to specify one or more extra images or
# other source files which should be copied to the HTML output directory. Note
# that these files will be copied to the base HTML output directory. Use the
# $relpath^ marker in the HTML_HEADER and/or HTML_FOOTER files to load these
# files. In the HTML_STYLESHEET file, use the file name only. Also note that the
# files will be copied as-is; there are no commands or markers available.
# This tag requires that the tag GENERATE_HTML is set to YES.
HTML_EXTRA_FILES =
# The HTML_COLORSTYLE_HUE tag controls the color of the HTML output. Doxygen
# will adjust the colors in the style sheet and background images according to
# this color. Hue is specified as an angle on a colorwheel, see
# http://en.wikipedia.org/wiki/Hue for more information. For instance the value
# 0 represents red, 60 is yellow, 120 is green, 180 is cyan, 240 is blue, 300
# purple, and 360 is red again.
# Minimum value: 0, maximum value: 359, default value: 220.
# This tag requires that the tag GENERATE_HTML is set to YES.
HTML_COLORSTYLE_HUE = 220
# The HTML_COLORSTYLE_SAT tag controls the purity (or saturation) of the colors
# in the HTML output. For a value of 0 the output will use grayscales only. A
# value of 255 will produce the most vivid colors.
# Minimum value: 0, maximum value: 255, default value: 100.
# This tag requires that the tag GENERATE_HTML is set to YES.
HTML_COLORSTYLE_SAT = 100
# The HTML_COLORSTYLE_GAMMA tag controls the gamma correction applied to the
# luminance component of the colors in the HTML output. Values below 100
# gradually make the output lighter, whereas values above 100 make the output
# darker. The value divided by 100 is the actual gamma applied, so 80 represents
# a gamma of 0.8, The value 220 represents a gamma of 2.2, and 100 does not
# change the gamma.
# Minimum value: 40, maximum value: 240, default value: 80.
# This tag requires that the tag GENERATE_HTML is set to YES.
HTML_COLORSTYLE_GAMMA = 80
# If the HTML_TIMESTAMP tag is set to YES then the footer of each generated HTML
# page will contain the date and time when the page was generated. Setting this
# to YES can help to show when doxygen was last run and thus if the
# documentation is up to date.
# The default value is: NO.
# This tag requires that the tag GENERATE_HTML is set to YES.
HTML_TIMESTAMP = YES
# If the HTML_DYNAMIC_SECTIONS tag is set to YES then the generated HTML
# documentation will contain sections that can be hidden and shown after the
# page has loaded.
# The default value is: NO.
# This tag requires that the tag GENERATE_HTML is set to YES.
HTML_DYNAMIC_SECTIONS = NO
# With HTML_INDEX_NUM_ENTRIES one can control the preferred number of entries
# shown in the various tree structured indices initially; the user can expand
# and collapse entries dynamically later on. Doxygen will expand the tree to
# such a level that at most the specified number of entries are visible (unless
# a fully collapsed tree already exceeds this amount). So setting the number of
# entries 1 will produce a full collapsed tree by default. 0 is a special value
# representing an infinite number of entries and will result in a full expanded
# tree by default.
# Minimum value: 0, maximum value: 9999, default value: 100.
# This tag requires that the tag GENERATE_HTML is set to YES.
HTML_INDEX_NUM_ENTRIES = 100
# If the GENERATE_DOCSET tag is set to YES, additional index files will be
# generated that can be used as input for Apple's Xcode 3 integrated development
# environment (see: http://developer.apple.com/tools/xcode/), introduced with
# OSX 10.5 (Leopard). To create a documentation set, doxygen will generate a
# Makefile in the HTML output directory. Running make will produce the docset in
# that directory and running make install will install the docset in
# ~/Library/Developer/Shared/Documentation/DocSets so that Xcode will find it at
# startup. See http://developer.apple.com/tools/creatingdocsetswithdoxygen.html
# for more information.
# The default value is: NO.
# This tag requires that the tag GENERATE_HTML is set to YES.
GENERATE_DOCSET = NO
# This tag determines the name of the docset feed. A documentation feed provides
# an umbrella under which multiple documentation sets from a single provider
# (such as a company or product suite) can be grouped.
# The default value is: Doxygen generated docs.
# This tag requires that the tag GENERATE_DOCSET is set to YES.
DOCSET_FEEDNAME = "librdkafka documentation"
# This tag specifies a string that should uniquely identify the documentation
# set bundle. This should be a reverse domain-name style string, e.g.
# com.mycompany.MyDocSet. Doxygen will append .docset to the name.
# The default value is: org.doxygen.Project.
# This tag requires that the tag GENERATE_DOCSET is set to YES.
DOCSET_BUNDLE_ID = se.edenhill.librdkafka
# The DOCSET_PUBLISHER_ID tag specifies a string that should uniquely identify
# the documentation publisher. This should be a reverse domain-name style
# string, e.g. com.mycompany.MyDocSet.documentation.
# The default value is: org.doxygen.Publisher.
# This tag requires that the tag GENERATE_DOCSET is set to YES.
DOCSET_PUBLISHER_ID = se.edenhill
# The DOCSET_PUBLISHER_NAME tag identifies the documentation publisher.
# The default value is: Publisher.
# This tag requires that the tag GENERATE_DOCSET is set to YES.
DOCSET_PUBLISHER_NAME = Magnus Edenhill
# If the GENERATE_HTMLHELP tag is set to YES then doxygen generates three
# additional HTML index files: index.hhp, index.hhc, and index.hhk. The
# index.hhp is a project file that can be read by Microsoft's HTML Help Workshop
# (see: http://www.microsoft.com/en-us/download/details.aspx?id=21138) on
# Windows.
#
# The HTML Help Workshop contains a compiler that can convert all HTML output
# generated by doxygen into a single compiled HTML file (.chm). Compiled HTML
# files are now used as the Windows 98 help format, and will replace the old
# Windows help format (.hlp) on all Windows platforms in the future. Compressed
# HTML files also contain an index, a table of contents, and you can search for
# words in the documentation. The HTML workshop also contains a viewer for
# compressed HTML files.
# The default value is: NO.
# This tag requires that the tag GENERATE_HTML is set to YES.
GENERATE_HTMLHELP = NO
# The CHM_FILE tag can be used to specify the file name of the resulting .chm
# file. You can add a path in front of the file if the result should not be
# written to the html output directory.
# This tag requires that the tag GENERATE_HTMLHELP is set to YES.
CHM_FILE =
# The HHC_LOCATION tag can be used to specify the location (absolute path
# including file name) of the HTML help compiler (hhc.exe). If non-empty,
# doxygen will try to run the HTML help compiler on the generated index.hhp.
# The file has to be specified with full path.
# This tag requires that the tag GENERATE_HTMLHELP is set to YES.
HHC_LOCATION =
# The GENERATE_CHI flag controls if a separate .chi index file is generated
# (YES) or that it should be included in the master .chm file (NO).
# The default value is: NO.
# This tag requires that the tag GENERATE_HTMLHELP is set to YES.
GENERATE_CHI = NO
# The CHM_INDEX_ENCODING is used to encode HtmlHelp index (hhk), content (hhc)
# and project file content.
# This tag requires that the tag GENERATE_HTMLHELP is set to YES.
CHM_INDEX_ENCODING =
# The BINARY_TOC flag controls whether a binary table of contents is generated
# (YES) or a normal table of contents (NO) in the .chm file. Furthermore it
# enables the Previous and Next buttons.
# The default value is: NO.
# This tag requires that the tag GENERATE_HTMLHELP is set to YES.
BINARY_TOC = NO
# The TOC_EXPAND flag can be set to YES to add extra items for group members to
# the table of contents of the HTML help documentation and to the tree view.
# The default value is: NO.
# This tag requires that the tag GENERATE_HTMLHELP is set to YES.
TOC_EXPAND = NO
# If the GENERATE_QHP tag is set to YES and both QHP_NAMESPACE and
# QHP_VIRTUAL_FOLDER are set, an additional index file will be generated that
# can be used as input for Qt's qhelpgenerator to generate a Qt Compressed Help
# (.qch) of the generated HTML documentation.
# The default value is: NO.
# This tag requires that the tag GENERATE_HTML is set to YES.
GENERATE_QHP = NO
# If the QHG_LOCATION tag is specified, the QCH_FILE tag can be used to specify
# the file name of the resulting .qch file. The path specified is relative to
# the HTML output folder.
# This tag requires that the tag GENERATE_QHP is set to YES.
QCH_FILE =
# The QHP_NAMESPACE tag specifies the namespace to use when generating Qt Help
# Project output. For more information please see Qt Help Project / Namespace
# (see: http://qt-project.org/doc/qt-4.8/qthelpproject.html#namespace).
# The default value is: org.doxygen.Project.
# This tag requires that the tag GENERATE_QHP is set to YES.
QHP_NAMESPACE = se.edenhill.librdkafka
# The QHP_VIRTUAL_FOLDER tag specifies the namespace to use when generating Qt
# Help Project output. For more information please see Qt Help Project / Virtual
# Folders (see: http://qt-project.org/doc/qt-4.8/qthelpproject.html#virtual-
# folders).
# The default value is: doc.
# This tag requires that the tag GENERATE_QHP is set to YES.
QHP_VIRTUAL_FOLDER = doc
# If the QHP_CUST_FILTER_NAME tag is set, it specifies the name of a custom
# filter to add. For more information please see Qt Help Project / Custom
# Filters (see: http://qt-project.org/doc/qt-4.8/qthelpproject.html#custom-
# filters).
# This tag requires that the tag GENERATE_QHP is set to YES.
QHP_CUST_FILTER_NAME =
# The QHP_CUST_FILTER_ATTRS tag specifies the list of the attributes of the
# custom filter to add. For more information please see Qt Help Project / Custom
# Filters (see: http://qt-project.org/doc/qt-4.8/qthelpproject.html#custom-
# filters).
# This tag requires that the tag GENERATE_QHP is set to YES.
QHP_CUST_FILTER_ATTRS =
# The QHP_SECT_FILTER_ATTRS tag specifies the list of the attributes this
# project's filter section matches. Qt Help Project / Filter Attributes (see:
# http://qt-project.org/doc/qt-4.8/qthelpproject.html#filter-attributes).
# This tag requires that the tag GENERATE_QHP is set to YES.
QHP_SECT_FILTER_ATTRS =
# The QHG_LOCATION tag can be used to specify the location of Qt's
# qhelpgenerator. If non-empty doxygen will try to run qhelpgenerator on the
# generated .qhp file.
# This tag requires that the tag GENERATE_QHP is set to YES.
QHG_LOCATION =
# If the GENERATE_ECLIPSEHELP tag is set to YES, additional index files will be
# generated, together with the HTML files, they form an Eclipse help plugin. To
# install this plugin and make it available under the help contents menu in
# Eclipse, the contents of the directory containing the HTML and XML files needs
# to be copied into the plugins directory of eclipse. The name of the directory
# within the plugins directory should be the same as the ECLIPSE_DOC_ID value.
# After copying Eclipse needs to be restarted before the help appears.
# The default value is: NO.
# This tag requires that the tag GENERATE_HTML is set to YES.
GENERATE_ECLIPSEHELP = NO
# A unique identifier for the Eclipse help plugin. When installing the plugin
# the directory name containing the HTML and XML files should also have this
# name. Each documentation set should have its own identifier.
# The default value is: org.doxygen.Project.
# This tag requires that the tag GENERATE_ECLIPSEHELP is set to YES.
ECLIPSE_DOC_ID = se.edenhill.librdkafka
# If you want full control over the layout of the generated HTML pages it might
# be necessary to disable the index and replace it with your own. The
# DISABLE_INDEX tag can be used to turn on/off the condensed index (tabs) at top
# of each HTML page. A value of NO enables the index and the value YES disables
# it. Since the tabs in the index contain the same information as the navigation
# tree, you can set this option to YES if you also set GENERATE_TREEVIEW to YES.
# The default value is: NO.
# This tag requires that the tag GENERATE_HTML is set to YES.
DISABLE_INDEX = NO
# The GENERATE_TREEVIEW tag is used to specify whether a tree-like index
# structure should be generated to display hierarchical information. If the tag
# value is set to YES, a side panel will be generated containing a tree-like
# index structure (just like the one that is generated for HTML Help). For this
# to work a browser that supports JavaScript, DHTML, CSS and frames is required
# (i.e. any modern browser). Windows users are probably better off using the
# HTML help feature. Via custom style sheets (see HTML_EXTRA_STYLESHEET) one can
# further fine-tune the look of the index. As an example, the default style
# sheet generated by doxygen has an example that shows how to put an image at
# the root of the tree instead of the PROJECT_NAME. Since the tree basically has
# the same information as the tab index, you could consider setting
# DISABLE_INDEX to YES when enabling this option.
# The default value is: NO.
# This tag requires that the tag GENERATE_HTML is set to YES.
GENERATE_TREEVIEW = YES
# The ENUM_VALUES_PER_LINE tag can be used to set the number of enum values that
# doxygen will group on one line in the generated HTML documentation.
#
# Note that a value of 0 will completely suppress the enum values from appearing
# in the overview section.
# Minimum value: 0, maximum value: 20, default value: 4.
# This tag requires that the tag GENERATE_HTML is set to YES.
ENUM_VALUES_PER_LINE = 1
# If the treeview is enabled (see GENERATE_TREEVIEW) then this tag can be used
# to set the initial width (in pixels) of the frame in which the tree is shown.
# Minimum value: 0, maximum value: 1500, default value: 250.
# This tag requires that the tag GENERATE_HTML is set to YES.
TREEVIEW_WIDTH = 250
# If the EXT_LINKS_IN_WINDOW option is set to YES, doxygen will open links to
# external symbols imported via tag files in a separate window.
# The default value is: NO.
# This tag requires that the tag GENERATE_HTML is set to YES.
EXT_LINKS_IN_WINDOW = NO
# Use this tag to change the font size of LaTeX formulas included as images in
# the HTML documentation. When you change the font size after a successful
# doxygen run you need to manually remove any form_*.png images from the HTML
# output directory to force them to be regenerated.
# Minimum value: 8, maximum value: 50, default value: 10.
# This tag requires that the tag GENERATE_HTML is set to YES.
FORMULA_FONTSIZE = 10
# Use the FORMULA_TRANPARENT tag to determine whether or not the images
# generated for formulas are transparent PNGs. Transparent PNGs are not
# supported properly for IE 6.0, but are supported on all modern browsers.
#
# Note that when changing this option you need to delete any form_*.png files in
# the HTML output directory before the changes have effect.
# The default value is: YES.
# This tag requires that the tag GENERATE_HTML is set to YES.
FORMULA_TRANSPARENT = YES
# Enable the USE_MATHJAX option to render LaTeX formulas using MathJax (see
# http://www.mathjax.org) which uses client side Javascript for the rendering
# instead of using pre-rendered bitmaps. Use this if you do not have LaTeX
# installed or if you want to formulas look prettier in the HTML output. When
# enabled you may also need to install MathJax separately and configure the path
# to it using the MATHJAX_RELPATH option.
# The default value is: NO.
# This tag requires that the tag GENERATE_HTML is set to YES.
USE_MATHJAX = NO
# When MathJax is enabled you can set the default output format to be used for
# the MathJax output. See the MathJax site (see:
# http://docs.mathjax.org/en/latest/output.html) for more details.
# Possible values are: HTML-CSS (which is slower, but has the best
# compatibility), NativeMML (i.e. MathML) and SVG.
# The default value is: HTML-CSS.
# This tag requires that the tag USE_MATHJAX is set to YES.
MATHJAX_FORMAT = HTML-CSS
# When MathJax is enabled you need to specify the location relative to the HTML
# output directory using the MATHJAX_RELPATH option. The destination directory
# should contain the MathJax.js script. For instance, if the mathjax directory
# is located at the same level as the HTML output directory, then
# MATHJAX_RELPATH should be ../mathjax. The default value points to the MathJax
# Content Delivery Network so you can quickly see the result without installing
# MathJax. However, it is strongly recommended to install a local copy of
# MathJax from http://www.mathjax.org before deployment.
# The default value is: http://cdn.mathjax.org/mathjax/latest.
# This tag requires that the tag USE_MATHJAX is set to YES.
MATHJAX_RELPATH = http://cdn.mathjax.org/mathjax/latest
# The MATHJAX_EXTENSIONS tag can be used to specify one or more MathJax
# extension names that should be enabled during MathJax rendering. For example
# MATHJAX_EXTENSIONS = TeX/AMSmath TeX/AMSsymbols
# This tag requires that the tag USE_MATHJAX is set to YES.
MATHJAX_EXTENSIONS =
# The MATHJAX_CODEFILE tag can be used to specify a file with javascript pieces
# of code that will be used on startup of the MathJax code. See the MathJax site
# (see: http://docs.mathjax.org/en/latest/output.html) for more details. For an
# example see the documentation.
# This tag requires that the tag USE_MATHJAX is set to YES.
MATHJAX_CODEFILE =
# When the SEARCHENGINE tag is enabled doxygen will generate a search box for
# the HTML output. The underlying search engine uses javascript and DHTML and
# should work on any modern browser. Note that when using HTML help
# (GENERATE_HTMLHELP), Qt help (GENERATE_QHP), or docsets (GENERATE_DOCSET)
# there is already a search function so this one should typically be disabled.
# For large projects the javascript based search engine can be slow, then
# enabling SERVER_BASED_SEARCH may provide a better solution. It is possible to
# search using the keyboard; to jump to the search box use + S
# (what the is depends on the OS and browser, but it is typically
# , /, or both). Inside the search box use the to jump into the search results window, the results can be navigated
# using the . Press to select an item or to cancel
# the search. The filter options can be selected when the cursor is inside the
# search box by pressing +. Also here use the
# to select a filter and or to activate or cancel the filter
# option.
# The default value is: YES.
# This tag requires that the tag GENERATE_HTML is set to YES.
SEARCHENGINE = YES
# When the SERVER_BASED_SEARCH tag is enabled the search engine will be
# implemented using a web server instead of a web client using Javascript. There
# are two flavors of web server based searching depending on the EXTERNAL_SEARCH
# setting. When disabled, doxygen will generate a PHP script for searching and
# an index file used by the script. When EXTERNAL_SEARCH is enabled the indexing
# and searching needs to be provided by external tools. See the section
# "External Indexing and Searching" for details.
# The default value is: NO.
# This tag requires that the tag SEARCHENGINE is set to YES.
SERVER_BASED_SEARCH = NO
# When EXTERNAL_SEARCH tag is enabled doxygen will no longer generate the PHP
# script for searching. Instead the search results are written to an XML file
# which needs to be processed by an external indexer. Doxygen will invoke an
# external search engine pointed to by the SEARCHENGINE_URL option to obtain the
# search results.
#
# Doxygen ships with an example indexer (doxyindexer) and search engine
# (doxysearch.cgi) which are based on the open source search engine library
# Xapian (see: http://xapian.org/).
#
# See the section "External Indexing and Searching" for details.
# The default value is: NO.
# This tag requires that the tag SEARCHENGINE is set to YES.
EXTERNAL_SEARCH = NO
# The SEARCHENGINE_URL should point to a search engine hosted by a web server
# which will return the search results when EXTERNAL_SEARCH is enabled.
#
# Doxygen ships with an example indexer (doxyindexer) and search engine
# (doxysearch.cgi) which are based on the open source search engine library
# Xapian (see: http://xapian.org/). See the section "External Indexing and
# Searching" for details.
# This tag requires that the tag SEARCHENGINE is set to YES.
SEARCHENGINE_URL =
# When SERVER_BASED_SEARCH and EXTERNAL_SEARCH are both enabled the unindexed
# search data is written to a file for indexing by an external tool. With the
# SEARCHDATA_FILE tag the name of this file can be specified.
# The default file is: searchdata.xml.
# This tag requires that the tag SEARCHENGINE is set to YES.
SEARCHDATA_FILE = searchdata.xml
# When SERVER_BASED_SEARCH and EXTERNAL_SEARCH are both enabled the
# EXTERNAL_SEARCH_ID tag can be used as an identifier for the project. This is
# useful in combination with EXTRA_SEARCH_MAPPINGS to search through multiple
# projects and redirect the results back to the right project.
# This tag requires that the tag SEARCHENGINE is set to YES.
EXTERNAL_SEARCH_ID =
# The EXTRA_SEARCH_MAPPINGS tag can be used to enable searching through doxygen
# projects other than the one defined by this configuration file, but that are
# all added to the same external search index. Each project needs to have a
# unique id set via EXTERNAL_SEARCH_ID. The search mapping then maps the id of
# to a relative location where the documentation can be found. The format is:
# EXTRA_SEARCH_MAPPINGS = tagname1=loc1 tagname2=loc2 ...
# This tag requires that the tag SEARCHENGINE is set to YES.
EXTRA_SEARCH_MAPPINGS =
#---------------------------------------------------------------------------
# Configuration options related to the LaTeX output
#---------------------------------------------------------------------------
# If the GENERATE_LATEX tag is set to YES, doxygen will generate LaTeX output.
# The default value is: YES.
GENERATE_LATEX = YES
# The LATEX_OUTPUT tag is used to specify where the LaTeX docs will be put. If a
# relative path is entered the value of OUTPUT_DIRECTORY will be put in front of
# it.
# The default directory is: latex.
# This tag requires that the tag GENERATE_LATEX is set to YES.
LATEX_OUTPUT = latex
# The LATEX_CMD_NAME tag can be used to specify the LaTeX command name to be
# invoked.
#
# Note that when enabling USE_PDFLATEX this option is only used for generating
# bitmaps for formulas in the HTML output, but not in the Makefile that is
# written to the output directory.
# The default file is: latex.
# This tag requires that the tag GENERATE_LATEX is set to YES.
LATEX_CMD_NAME = latex
# The MAKEINDEX_CMD_NAME tag can be used to specify the command name to generate
# index for LaTeX.
# The default file is: makeindex.
# This tag requires that the tag GENERATE_LATEX is set to YES.
MAKEINDEX_CMD_NAME = makeindex
# If the COMPACT_LATEX tag is set to YES, doxygen generates more compact LaTeX
# documents. This may be useful for small projects and may help to save some
# trees in general.
# The default value is: NO.
# This tag requires that the tag GENERATE_LATEX is set to YES.
COMPACT_LATEX = NO
# The PAPER_TYPE tag can be used to set the paper type that is used by the
# printer.
# Possible values are: a4 (210 x 297 mm), letter (8.5 x 11 inches), legal (8.5 x
# 14 inches) and executive (7.25 x 10.5 inches).
# The default value is: a4.
# This tag requires that the tag GENERATE_LATEX is set to YES.
PAPER_TYPE = a4
# The EXTRA_PACKAGES tag can be used to specify one or more LaTeX package names
# that should be included in the LaTeX output. To get the times font for
# instance you can specify
# EXTRA_PACKAGES=times
# If left blank no extra packages will be included.
# This tag requires that the tag GENERATE_LATEX is set to YES.
EXTRA_PACKAGES =
# The LATEX_HEADER tag can be used to specify a personal LaTeX header for the
# generated LaTeX document. The header should contain everything until the first
# chapter. If it is left blank doxygen will generate a standard header. See
# section "Doxygen usage" for information on how to let doxygen write the
# default header to a separate file.
#
# Note: Only use a user-defined header if you know what you are doing! The
# following commands have a special meaning inside the header: $title,
# $datetime, $date, $doxygenversion, $projectname, $projectnumber,
# $projectbrief, $projectlogo. Doxygen will replace $title with the empty
# string, for the replacement values of the other commands the user is referred
# to HTML_HEADER.
# This tag requires that the tag GENERATE_LATEX is set to YES.
LATEX_HEADER =
# The LATEX_FOOTER tag can be used to specify a personal LaTeX footer for the
# generated LaTeX document. The footer should contain everything after the last
# chapter. If it is left blank doxygen will generate a standard footer. See
# LATEX_HEADER for more information on how to generate a default footer and what
# special commands can be used inside the footer.
#
# Note: Only use a user-defined footer if you know what you are doing!
# This tag requires that the tag GENERATE_LATEX is set to YES.
LATEX_FOOTER =
# The LATEX_EXTRA_STYLESHEET tag can be used to specify additional user-defined
# LaTeX style sheets that are included after the standard style sheets created
# by doxygen. Using this option one can overrule certain style aspects. Doxygen
# will copy the style sheet files to the output directory.
# Note: The order of the extra style sheet files is of importance (e.g. the last
# style sheet in the list overrules the setting of the previous ones in the
# list).
# This tag requires that the tag GENERATE_LATEX is set to YES.
LATEX_EXTRA_STYLESHEET =
# The LATEX_EXTRA_FILES tag can be used to specify one or more extra images or
# other source files which should be copied to the LATEX_OUTPUT output
# directory. Note that the files will be copied as-is; there are no commands or
# markers available.
# This tag requires that the tag GENERATE_LATEX is set to YES.
LATEX_EXTRA_FILES =
# If the PDF_HYPERLINKS tag is set to YES, the LaTeX that is generated is
# prepared for conversion to PDF (using ps2pdf or pdflatex). The PDF file will
# contain links (just like the HTML output) instead of page references. This
# makes the output suitable for online browsing using a PDF viewer.
# The default value is: YES.
# This tag requires that the tag GENERATE_LATEX is set to YES.
PDF_HYPERLINKS = YES
# If the USE_PDFLATEX tag is set to YES, doxygen will use pdflatex to generate
# the PDF file directly from the LaTeX files. Set this option to YES, to get a
# higher quality PDF documentation.
# The default value is: YES.
# This tag requires that the tag GENERATE_LATEX is set to YES.
USE_PDFLATEX = YES
# If the LATEX_BATCHMODE tag is set to YES, doxygen will add the \batchmode
# command to the generated LaTeX files. This will instruct LaTeX to keep running
# if errors occur, instead of asking the user for help. This option is also used
# when generating formulas in HTML.
# The default value is: NO.
# This tag requires that the tag GENERATE_LATEX is set to YES.
LATEX_BATCHMODE = NO
# If the LATEX_HIDE_INDICES tag is set to YES then doxygen will not include the
# index chapters (such as File Index, Compound Index, etc.) in the output.
# The default value is: NO.
# This tag requires that the tag GENERATE_LATEX is set to YES.
LATEX_HIDE_INDICES = NO
# If the LATEX_SOURCE_CODE tag is set to YES then doxygen will include source
# code with syntax highlighting in the LaTeX output.
#
# Note that which sources are shown also depends on other settings such as
# SOURCE_BROWSER.
# The default value is: NO.
# This tag requires that the tag GENERATE_LATEX is set to YES.
LATEX_SOURCE_CODE = NO
# The LATEX_BIB_STYLE tag can be used to specify the style to use for the
# bibliography, e.g. plainnat, or ieeetr. See
# http://en.wikipedia.org/wiki/BibTeX and \cite for more info.
# The default value is: plain.
# This tag requires that the tag GENERATE_LATEX is set to YES.
LATEX_BIB_STYLE = plain
#---------------------------------------------------------------------------
# Configuration options related to the RTF output
#---------------------------------------------------------------------------
# If the GENERATE_RTF tag is set to YES, doxygen will generate RTF output. The
# RTF output is optimized for Word 97 and may not look too pretty with other RTF
# readers/editors.
# The default value is: NO.
GENERATE_RTF = NO
# The RTF_OUTPUT tag is used to specify where the RTF docs will be put. If a
# relative path is entered the value of OUTPUT_DIRECTORY will be put in front of
# it.
# The default directory is: rtf.
# This tag requires that the tag GENERATE_RTF is set to YES.
RTF_OUTPUT = rtf
# If the COMPACT_RTF tag is set to YES, doxygen generates more compact RTF
# documents. This may be useful for small projects and may help to save some
# trees in general.
# The default value is: NO.
# This tag requires that the tag GENERATE_RTF is set to YES.
COMPACT_RTF = NO
# If the RTF_HYPERLINKS tag is set to YES, the RTF that is generated will
# contain hyperlink fields. The RTF file will contain links (just like the HTML
# output) instead of page references. This makes the output suitable for online
# browsing using Word or some other Word compatible readers that support those
# fields.
#
# Note: WordPad (write) and others do not support links.
# The default value is: NO.
# This tag requires that the tag GENERATE_RTF is set to YES.
RTF_HYPERLINKS = NO
# Load stylesheet definitions from file. Syntax is similar to doxygen's config
# file, i.e. a series of assignments. You only have to provide replacements,
# missing definitions are set to their default value.
#
# See also section "Doxygen usage" for information on how to generate the
# default style sheet that doxygen normally uses.
# This tag requires that the tag GENERATE_RTF is set to YES.
RTF_STYLESHEET_FILE =
# Set optional variables used in the generation of an RTF document. Syntax is
# similar to doxygen's config file. A template extensions file can be generated
# using doxygen -e rtf extensionFile.
# This tag requires that the tag GENERATE_RTF is set to YES.
RTF_EXTENSIONS_FILE =
# If the RTF_SOURCE_CODE tag is set to YES then doxygen will include source code
# with syntax highlighting in the RTF output.
#
# Note that which sources are shown also depends on other settings such as
# SOURCE_BROWSER.
# The default value is: NO.
# This tag requires that the tag GENERATE_RTF is set to YES.
RTF_SOURCE_CODE = NO
#---------------------------------------------------------------------------
# Configuration options related to the man page output
#---------------------------------------------------------------------------
# If the GENERATE_MAN tag is set to YES, doxygen will generate man pages for
# classes and files.
# The default value is: NO.
GENERATE_MAN = NO
# The MAN_OUTPUT tag is used to specify where the man pages will be put. If a
# relative path is entered the value of OUTPUT_DIRECTORY will be put in front of
# it. A directory man3 will be created inside the directory specified by
# MAN_OUTPUT.
# The default directory is: man.
# This tag requires that the tag GENERATE_MAN is set to YES.
MAN_OUTPUT = man
# The MAN_EXTENSION tag determines the extension that is added to the generated
# man pages. In case the manual section does not start with a number, the number
# 3 is prepended. The dot (.) at the beginning of the MAN_EXTENSION tag is
# optional.
# The default value is: .3.
# This tag requires that the tag GENERATE_MAN is set to YES.
MAN_EXTENSION = .3
# The MAN_SUBDIR tag determines the name of the directory created within
# MAN_OUTPUT in which the man pages are placed. If defaults to man followed by
# MAN_EXTENSION with the initial . removed.
# This tag requires that the tag GENERATE_MAN is set to YES.
MAN_SUBDIR =
# If the MAN_LINKS tag is set to YES and doxygen generates man output, then it
# will generate one additional man file for each entity documented in the real
# man page(s). These additional files only source the real man page, but without
# them the man command would be unable to find the correct page.
# The default value is: NO.
# This tag requires that the tag GENERATE_MAN is set to YES.
MAN_LINKS = NO
#---------------------------------------------------------------------------
# Configuration options related to the XML output
#---------------------------------------------------------------------------
# If the GENERATE_XML tag is set to YES, doxygen will generate an XML file that
# captures the structure of the code including all documentation.
# The default value is: NO.
GENERATE_XML = NO
# The XML_OUTPUT tag is used to specify where the XML pages will be put. If a
# relative path is entered the value of OUTPUT_DIRECTORY will be put in front of
# it.
# The default directory is: xml.
# This tag requires that the tag GENERATE_XML is set to YES.
XML_OUTPUT = xml
# If the XML_PROGRAMLISTING tag is set to YES, doxygen will dump the program
# listings (including syntax highlighting and cross-referencing information) to
# the XML output. Note that enabling this will significantly increase the size
# of the XML output.
# The default value is: YES.
# This tag requires that the tag GENERATE_XML is set to YES.
XML_PROGRAMLISTING = YES
#---------------------------------------------------------------------------
# Configuration options related to the DOCBOOK output
#---------------------------------------------------------------------------
# If the GENERATE_DOCBOOK tag is set to YES, doxygen will generate Docbook files
# that can be used to generate PDF.
# The default value is: NO.
GENERATE_DOCBOOK = NO
# The DOCBOOK_OUTPUT tag is used to specify where the Docbook pages will be put.
# If a relative path is entered the value of OUTPUT_DIRECTORY will be put in
# front of it.
# The default directory is: docbook.
# This tag requires that the tag GENERATE_DOCBOOK is set to YES.
DOCBOOK_OUTPUT = docbook
# If the DOCBOOK_PROGRAMLISTING tag is set to YES, doxygen will include the
# program listings (including syntax highlighting and cross-referencing
# information) to the DOCBOOK output. Note that enabling this will significantly
# increase the size of the DOCBOOK output.
# The default value is: NO.
# This tag requires that the tag GENERATE_DOCBOOK is set to YES.
DOCBOOK_PROGRAMLISTING = NO
#---------------------------------------------------------------------------
# Configuration options for the AutoGen Definitions output
#---------------------------------------------------------------------------
# If the GENERATE_AUTOGEN_DEF tag is set to YES, doxygen will generate an
# AutoGen Definitions (see http://autogen.sf.net) file that captures the
# structure of the code including all documentation. Note that this feature is
# still experimental and incomplete at the moment.
# The default value is: NO.
GENERATE_AUTOGEN_DEF = NO
#---------------------------------------------------------------------------
# Configuration options related to the Perl module output
#---------------------------------------------------------------------------
# If the GENERATE_PERLMOD tag is set to YES, doxygen will generate a Perl module
# file that captures the structure of the code including all documentation.
#
# Note that this feature is still experimental and incomplete at the moment.
# The default value is: NO.
GENERATE_PERLMOD = NO
# If the PERLMOD_LATEX tag is set to YES, doxygen will generate the necessary
# Makefile rules, Perl scripts and LaTeX code to be able to generate PDF and DVI
# output from the Perl module output.
# The default value is: NO.
# This tag requires that the tag GENERATE_PERLMOD is set to YES.
PERLMOD_LATEX = NO
# If the PERLMOD_PRETTY tag is set to YES, the Perl module output will be nicely
# formatted so it can be parsed by a human reader. This is useful if you want to
# understand what is going on. On the other hand, if this tag is set to NO, the
# size of the Perl module output will be much smaller and Perl will parse it
# just the same.
# The default value is: YES.
# This tag requires that the tag GENERATE_PERLMOD is set to YES.
PERLMOD_PRETTY = YES
# The names of the make variables in the generated doxyrules.make file are
# prefixed with the string contained in PERLMOD_MAKEVAR_PREFIX. This is useful
# so different doxyrules.make files included by the same Makefile don't
# overwrite each other's variables.
# This tag requires that the tag GENERATE_PERLMOD is set to YES.
PERLMOD_MAKEVAR_PREFIX =
#---------------------------------------------------------------------------
# Configuration options related to the preprocessor
#---------------------------------------------------------------------------
# If the ENABLE_PREPROCESSING tag is set to YES, doxygen will evaluate all
# C-preprocessor directives found in the sources and include files.
# The default value is: YES.
ENABLE_PREPROCESSING = YES
# If the MACRO_EXPANSION tag is set to YES, doxygen will expand all macro names
# in the source code. If set to NO, only conditional compilation will be
# performed. Macro expansion can be done in a controlled way by setting
# EXPAND_ONLY_PREDEF to YES.
# The default value is: NO.
# This tag requires that the tag ENABLE_PREPROCESSING is set to YES.
MACRO_EXPANSION = NO
# If the EXPAND_ONLY_PREDEF and MACRO_EXPANSION tags are both set to YES then
# the macro expansion is limited to the macros specified with the PREDEFINED and
# EXPAND_AS_DEFINED tags.
# The default value is: NO.
# This tag requires that the tag ENABLE_PREPROCESSING is set to YES.
EXPAND_ONLY_PREDEF = NO
# If the SEARCH_INCLUDES tag is set to YES, the include files in the
# INCLUDE_PATH will be searched if a #include is found.
# The default value is: YES.
# This tag requires that the tag ENABLE_PREPROCESSING is set to YES.
SEARCH_INCLUDES = YES
# The INCLUDE_PATH tag can be used to specify one or more directories that
# contain include files that are not input files but should be processed by the
# preprocessor.
# This tag requires that the tag SEARCH_INCLUDES is set to YES.
INCLUDE_PATH =
# You can use the INCLUDE_FILE_PATTERNS tag to specify one or more wildcard
# patterns (like *.h and *.hpp) to filter out the header-files in the
# directories. If left blank, the patterns specified with FILE_PATTERNS will be
# used.
# This tag requires that the tag ENABLE_PREPROCESSING is set to YES.
INCLUDE_FILE_PATTERNS =
# The PREDEFINED tag can be used to specify one or more macro names that are
# defined before the preprocessor is started (similar to the -D option of e.g.
# gcc). The argument of the tag is a list of macros of the form: name or
# name=definition (no spaces). If the definition and the "=" are omitted, "=1"
# is assumed. To prevent a macro definition from being undefined via #undef or
# recursively expanded use the := operator instead of the = operator.
# This tag requires that the tag ENABLE_PREPROCESSING is set to YES.
PREDEFINED =
# If the MACRO_EXPANSION and EXPAND_ONLY_PREDEF tags are set to YES then this
# tag can be used to specify a list of macro names that should be expanded. The
# macro definition that is found in the sources will be used. Use the PREDEFINED
# tag if you want to use a different macro definition that overrules the
# definition found in the source code.
# This tag requires that the tag ENABLE_PREPROCESSING is set to YES.
EXPAND_AS_DEFINED =
# If the SKIP_FUNCTION_MACROS tag is set to YES then doxygen's preprocessor will
# remove all references to function-like macros that are alone on a line, have
# an all uppercase name, and do not end with a semicolon. Such function macros
# are typically used for boiler-plate code, and will confuse the parser if not
# removed.
# The default value is: YES.
# This tag requires that the tag ENABLE_PREPROCESSING is set to YES.
SKIP_FUNCTION_MACROS = NO
#---------------------------------------------------------------------------
# Configuration options related to external references
#---------------------------------------------------------------------------
# The TAGFILES tag can be used to specify one or more tag files. For each tag
# file the location of the external documentation should be added. The format of
# a tag file without this location is as follows:
# TAGFILES = file1 file2 ...
# Adding location for the tag files is done as follows:
# TAGFILES = file1=loc1 "file2 = loc2" ...
# where loc1 and loc2 can be relative or absolute paths or URLs. See the
# section "Linking to external documentation" for more information about the use
# of tag files.
# Note: Each tag file must have a unique name (where the name does NOT include
# the path). If a tag file is not located in the directory in which doxygen is
# run, you must also specify the path to the tagfile here.
TAGFILES =
# When a file name is specified after GENERATE_TAGFILE, doxygen will create a
# tag file that is based on the input files it reads. See section "Linking to
# external documentation" for more information about the usage of tag files.
GENERATE_TAGFILE =
# If the ALLEXTERNALS tag is set to YES, all external class will be listed in
# the class index. If set to NO, only the inherited external classes will be
# listed.
# The default value is: NO.
ALLEXTERNALS = NO
# If the EXTERNAL_GROUPS tag is set to YES, all external groups will be listed
# in the modules index. If set to NO, only the current project's groups will be
# listed.
# The default value is: YES.
EXTERNAL_GROUPS = YES
# If the EXTERNAL_PAGES tag is set to YES, all external pages will be listed in
# the related pages index. If set to NO, only the current project's pages will
# be listed.
# The default value is: YES.
EXTERNAL_PAGES = YES
# The PERL_PATH should be the absolute path and name of the perl script
# interpreter (i.e. the result of 'which perl').
# The default file (with absolute path) is: /usr/bin/perl.
PERL_PATH = /usr/bin/perl
#---------------------------------------------------------------------------
# Configuration options related to the dot tool
#---------------------------------------------------------------------------
# If the CLASS_DIAGRAMS tag is set to YES, doxygen will generate a class diagram
# (in HTML and LaTeX) for classes with base or super classes. Setting the tag to
# NO turns the diagrams off. Note that this option also works with HAVE_DOT
# disabled, but it is recommended to install and use dot, since it yields more
# powerful graphs.
# The default value is: YES.
CLASS_DIAGRAMS = YES
# You can define message sequence charts within doxygen comments using the \msc
# command. Doxygen will then run the mscgen tool (see:
# http://www.mcternan.me.uk/mscgen/)) to produce the chart and insert it in the
# documentation. The MSCGEN_PATH tag allows you to specify the directory where
# the mscgen tool resides. If left empty the tool is assumed to be found in the
# default search path.
MSCGEN_PATH =
# You can include diagrams made with dia in doxygen documentation. Doxygen will
# then run dia to produce the diagram and insert it in the documentation. The
# DIA_PATH tag allows you to specify the directory where the dia binary resides.
# If left empty dia is assumed to be found in the default search path.
DIA_PATH =
# If set to YES the inheritance and collaboration graphs will hide inheritance
# and usage relations if the target is undocumented or is not a class.
# The default value is: YES.
HIDE_UNDOC_RELATIONS = YES
# If you set the HAVE_DOT tag to YES then doxygen will assume the dot tool is
# available from the path. This tool is part of Graphviz (see:
# http://www.graphviz.org/), a graph visualization toolkit from AT&T and Lucent
# Bell Labs. The other options in this section have no effect if this option is
# set to NO
# The default value is: YES.
HAVE_DOT = YES
# The DOT_NUM_THREADS specifies the number of dot invocations doxygen is allowed
# to run in parallel. When set to 0 doxygen will base this on the number of
# processors available in the system. You can set it explicitly to a value
# larger than 0 to get control over the balance between CPU load and processing
# speed.
# Minimum value: 0, maximum value: 32, default value: 0.
# This tag requires that the tag HAVE_DOT is set to YES.
DOT_NUM_THREADS = 0
# When you want a differently looking font in the dot files that doxygen
# generates you can specify the font name using DOT_FONTNAME. You need to make
# sure dot is able to find the font, which can be done by putting it in a
# standard location or by setting the DOTFONTPATH environment variable or by
# setting DOT_FONTPATH to the directory containing the font.
# The default value is: Helvetica.
# This tag requires that the tag HAVE_DOT is set to YES.
DOT_FONTNAME = Helvetica
# The DOT_FONTSIZE tag can be used to set the size (in points) of the font of
# dot graphs.
# Minimum value: 4, maximum value: 24, default value: 10.
# This tag requires that the tag HAVE_DOT is set to YES.
DOT_FONTSIZE = 10
# By default doxygen will tell dot to use the default font as specified with
# DOT_FONTNAME. If you specify a different font using DOT_FONTNAME you can set
# the path where dot can find it using this tag.
# This tag requires that the tag HAVE_DOT is set to YES.
DOT_FONTPATH =
# If the CLASS_GRAPH tag is set to YES then doxygen will generate a graph for
# each documented class showing the direct and indirect inheritance relations.
# Setting this tag to YES will force the CLASS_DIAGRAMS tag to NO.
# The default value is: YES.
# This tag requires that the tag HAVE_DOT is set to YES.
CLASS_GRAPH = YES
# If the COLLABORATION_GRAPH tag is set to YES then doxygen will generate a
# graph for each documented class showing the direct and indirect implementation
# dependencies (inheritance, containment, and class references variables) of the
# class with other documented classes.
# The default value is: YES.
# This tag requires that the tag HAVE_DOT is set to YES.
COLLABORATION_GRAPH = YES
# If the GROUP_GRAPHS tag is set to YES then doxygen will generate a graph for
# groups, showing the direct groups dependencies.
# The default value is: YES.
# This tag requires that the tag HAVE_DOT is set to YES.
GROUP_GRAPHS = YES
# If the UML_LOOK tag is set to YES, doxygen will generate inheritance and
# collaboration diagrams in a style similar to the OMG's Unified Modeling
# Language.
# The default value is: NO.
# This tag requires that the tag HAVE_DOT is set to YES.
UML_LOOK = NO
# If the UML_LOOK tag is enabled, the fields and methods are shown inside the
# class node. If there are many fields or methods and many nodes the graph may
# become too big to be useful. The UML_LIMIT_NUM_FIELDS threshold limits the
# number of items for each type to make the size more manageable. Set this to 0
# for no limit. Note that the threshold may be exceeded by 50% before the limit
# is enforced. So when you set the threshold to 10, up to 15 fields may appear,
# but if the number exceeds 15, the total amount of fields shown is limited to
# 10.
# Minimum value: 0, maximum value: 100, default value: 10.
# This tag requires that the tag HAVE_DOT is set to YES.
UML_LIMIT_NUM_FIELDS = 10
# If the TEMPLATE_RELATIONS tag is set to YES then the inheritance and
# collaboration graphs will show the relations between templates and their
# instances.
# The default value is: NO.
# This tag requires that the tag HAVE_DOT is set to YES.
TEMPLATE_RELATIONS = NO
# If the INCLUDE_GRAPH, ENABLE_PREPROCESSING and SEARCH_INCLUDES tags are set to
# YES then doxygen will generate a graph for each documented file showing the
# direct and indirect include dependencies of the file with other documented
# files.
# The default value is: YES.
# This tag requires that the tag HAVE_DOT is set to YES.
INCLUDE_GRAPH = YES
# If the INCLUDED_BY_GRAPH, ENABLE_PREPROCESSING and SEARCH_INCLUDES tags are
# set to YES then doxygen will generate a graph for each documented file showing
# the direct and indirect include dependencies of the file with other documented
# files.
# The default value is: YES.
# This tag requires that the tag HAVE_DOT is set to YES.
INCLUDED_BY_GRAPH = YES
# If the CALL_GRAPH tag is set to YES then doxygen will generate a call
# dependency graph for every global function or class method.
#
# Note that enabling this option will significantly increase the time of a run.
# So in most cases it will be better to enable call graphs for selected
# functions only using the \callgraph command.
# The default value is: NO.
# This tag requires that the tag HAVE_DOT is set to YES.
CALL_GRAPH = NO
# If the CALLER_GRAPH tag is set to YES then doxygen will generate a caller
# dependency graph for every global function or class method.
#
# Note that enabling this option will significantly increase the time of a run.
# So in most cases it will be better to enable caller graphs for selected
# functions only using the \callergraph command.
# The default value is: NO.
# This tag requires that the tag HAVE_DOT is set to YES.
CALLER_GRAPH = NO
# If the GRAPHICAL_HIERARCHY tag is set to YES then doxygen will graphical
# hierarchy of all classes instead of a textual one.
# The default value is: YES.
# This tag requires that the tag HAVE_DOT is set to YES.
GRAPHICAL_HIERARCHY = YES
# If the DIRECTORY_GRAPH tag is set to YES then doxygen will show the
# dependencies a directory has on other directories in a graphical way. The
# dependency relations are determined by the #include relations between the
# files in the directories.
# The default value is: YES.
# This tag requires that the tag HAVE_DOT is set to YES.
DIRECTORY_GRAPH = YES
# The DOT_IMAGE_FORMAT tag can be used to set the image format of the images
# generated by dot.
# Note: If you choose svg you need to set HTML_FILE_EXTENSION to xhtml in order
# to make the SVG files visible in IE 9+ (other browsers do not have this
# requirement).
# Possible values are: png, png:cairo, png:cairo:cairo, png:cairo:gd, png:gd,
# png:gd:gd, jpg, jpg:cairo, jpg:cairo:gd, jpg:gd, jpg:gd:gd, gif, gif:cairo,
# gif:cairo:gd, gif:gd, gif:gd:gd and svg.
# The default value is: png.
# This tag requires that the tag HAVE_DOT is set to YES.
DOT_IMAGE_FORMAT = png
# If DOT_IMAGE_FORMAT is set to svg, then this option can be set to YES to
# enable generation of interactive SVG images that allow zooming and panning.
#
# Note that this requires a modern browser other than Internet Explorer. Tested
# and working are Firefox, Chrome, Safari, and Opera.
# Note: For IE 9+ you need to set HTML_FILE_EXTENSION to xhtml in order to make
# the SVG files visible. Older versions of IE do not have SVG support.
# The default value is: NO.
# This tag requires that the tag HAVE_DOT is set to YES.
INTERACTIVE_SVG = NO
# The DOT_PATH tag can be used to specify the path where the dot tool can be
# found. If left blank, it is assumed the dot tool can be found in the path.
# This tag requires that the tag HAVE_DOT is set to YES.
DOT_PATH =
# The DOTFILE_DIRS tag can be used to specify one or more directories that
# contain dot files that are included in the documentation (see the \dotfile
# command).
# This tag requires that the tag HAVE_DOT is set to YES.
DOTFILE_DIRS =
# The MSCFILE_DIRS tag can be used to specify one or more directories that
# contain msc files that are included in the documentation (see the \mscfile
# command).
MSCFILE_DIRS =
# The DIAFILE_DIRS tag can be used to specify one or more directories that
# contain dia files that are included in the documentation (see the \diafile
# command).
DIAFILE_DIRS =
# When using plantuml, the PLANTUML_JAR_PATH tag should be used to specify the
# path where java can find the plantuml.jar file. If left blank, it is assumed
# PlantUML is not used or called during a preprocessing step. Doxygen will
# generate a warning when it encounters a \startuml command in this case and
# will not generate output for the diagram.
PLANTUML_JAR_PATH =
# When using plantuml, the specified paths are searched for files specified by
# the !include statement in a plantuml block.
PLANTUML_INCLUDE_PATH =
# The DOT_GRAPH_MAX_NODES tag can be used to set the maximum number of nodes
# that will be shown in the graph. If the number of nodes in a graph becomes
# larger than this value, doxygen will truncate the graph, which is visualized
# by representing a node as a red box. Note that doxygen if the number of direct
# children of the root node in a graph is already larger than
# DOT_GRAPH_MAX_NODES then the graph will not be shown at all. Also note that
# the size of a graph can be further restricted by MAX_DOT_GRAPH_DEPTH.
# Minimum value: 0, maximum value: 10000, default value: 50.
# This tag requires that the tag HAVE_DOT is set to YES.
DOT_GRAPH_MAX_NODES = 50
# The MAX_DOT_GRAPH_DEPTH tag can be used to set the maximum depth of the graphs
# generated by dot. A depth value of 3 means that only nodes reachable from the
# root by following a path via at most 3 edges will be shown. Nodes that lay
# further from the root node will be omitted. Note that setting this option to 1
# or 2 may greatly reduce the computation time needed for large code bases. Also
# note that the size of a graph can be further restricted by
# DOT_GRAPH_MAX_NODES. Using a depth of 0 means no depth restriction.
# Minimum value: 0, maximum value: 1000, default value: 0.
# This tag requires that the tag HAVE_DOT is set to YES.
MAX_DOT_GRAPH_DEPTH = 0
# Set the DOT_TRANSPARENT tag to YES to generate images with a transparent
# background. This is disabled by default, because dot on Windows does not seem
# to support this out of the box.
#
# Warning: Depending on the platform used, enabling this option may lead to
# badly anti-aliased labels on the edges of a graph (i.e. they become hard to
# read).
# The default value is: NO.
# This tag requires that the tag HAVE_DOT is set to YES.
DOT_TRANSPARENT = NO
# Set the DOT_MULTI_TARGETS tag to YES to allow dot to generate multiple output
# files in one run (i.e. multiple -o and -T options on the command line). This
# makes dot run faster, but since only newer versions of dot (>1.8.10) support
# this, this feature is disabled by default.
# The default value is: NO.
# This tag requires that the tag HAVE_DOT is set to YES.
DOT_MULTI_TARGETS = NO
# If the GENERATE_LEGEND tag is set to YES doxygen will generate a legend page
# explaining the meaning of the various boxes and arrows in the dot generated
# graphs.
# The default value is: YES.
# This tag requires that the tag HAVE_DOT is set to YES.
GENERATE_LEGEND = YES
# If the DOT_CLEANUP tag is set to YES, doxygen will remove the intermediate dot
# files that are used to generate the various graphs.
# The default value is: YES.
# This tag requires that the tag HAVE_DOT is set to YES.
DOT_CLEANUP = YES
librdkafka-0.9.3/INTRODUCTION.md 0000664 0000000 0000000 00000046412 13041406524 0016066 0 ustar 00root root 0000000 0000000 //@file INTRODUCTION.md
# Introduction to librdkafka - the Apache Kafka C/C++ client library
librdkafka is a high performance C implementation of the Apache
Kafka client, providing a reliable and performant client for production use.
librdkafka also provides a native C++ interface.
## Contents
The following chapters are available in this document
* Performance
* Performance numbers
* High throughput
* Low latency
* Compression
* Message reliability
* Usage
* Documentation
* Initialization
* Configuration
* Threads and callbacks
* Brokers
* Producer API
* Consumer API
* Appendix
* Test detailts
## Performance
librdkafka is a multi-threaded library designed for use on modern hardware and
it attempts to keep memory copying at a minimal. The payload of produced or
consumed messages may pass through without any copying
(if so desired by the application) putting no limit on message sizes.
librdkafka allows you to decide if high throughput is the name of the game,
or if a low latency service is required, all through the configuration
property interface.
The two most important configuration properties for performance tuning are:
* batch.num.messages - the minimum number of messages to wait for to
accumulate in the local queue before sending off a message set.
* queue.buffering.max.ms - how long to wait for batch.num.messages to
fill up in the local queue.
### Performance numbers
The following performance numbers stem from tests using the following setup:
* Intel Quad Core i7 at 3.4GHz, 8GB of memory
* Disk performance has been shortcut by setting the brokers' flush
configuration properties as so:
* `log.flush.interval.messages=10000000`
* `log.flush.interval.ms=100000`
* Two brokers running on the same machine as librdkafka.
* One topic with two partitions.
* Each broker is leader for one partition each.
* Using `rdkafka_performance` program available in the `examples` subdir.
**Test results**
* **Test1**: 2 brokers, 2 partitions, required.acks=2, 100 byte messages:
**850000 messages/second**, **85 MB/second**
* **Test2**: 1 broker, 1 partition, required.acks=0, 100 byte messages:
**710000 messages/second**, **71 MB/second**
* **Test3**: 2 broker2, 2 partitions, required.acks=2, 100 byte messages,
snappy compression:
**300000 messages/second**, **30 MB/second**
* **Test4**: 2 broker2, 2 partitions, required.acks=2, 100 byte messages,
gzip compression:
**230000 messages/second**, **23 MB/second**
**Note**: See the *Test details* chapter at the end of this document for
information about the commands executed, etc.
**Note**: Consumer performance tests will be announced soon.
### High throughput
The key to high throughput is message batching - waiting for a certain amount
of messages to accumulate in the local queue before sending them off in
one large message set or batch to the peer. This amortizes the messaging
overhead and eliminates the adverse effect of the round trip time (rtt).
The default settings, batch.num.messages=10000 and queue.buffering.max.ms=1000,
are suitable for high throughput. This allows librdkafka to wait up to
1000 ms for up to 1000 messages to accumulate in the local queue before
sending the accumulate messages to the broker.
These setting are set globally (`rd_kafka_conf_t`) but applies on a
per topic+partition basis.
### Low latency
When low latency messaging is required the "queue.buffering.max.ms" should be
tuned to the maximum permitted producer-side latency.
Setting queue.buffering.max.ms to 1 will make sure messages are sent as
soon as possible. You could check out [How to decrease message latency]
(https://github.com/edenhill/librdkafka/wiki/How-to-decrease-message-latency)
to find more details.
### Compression
Producer message compression is enabled through the "compression.codec"
configuration property.
Compression is performed on the batch of messages in the local queue, the
larger the batch the higher likelyhood of a higher compression ratio.
The local batch queue size is controlled through the "batch.num.messages" and
"queue.buffering.max.ms" configuration properties as described in the
**High throughput** chapter above.
## Message reliability
Message reliability is an important factor of librdkafka - an application
can rely fully on librdkafka to deliver a message according to the specified
configuration ("request.required.acks" and "message.send.max.retries", etc).
If the topic configuration property "request.required.acks" is set to wait
for message commit acknowledgements from brokers (any value but 0, see
[`CONFIGURATION.md`](https://github.com/edenhill/librdkafka/blob/master/CONFIGURATION.md)
for specifics) then librdkafka will hold on to the message until
all expected acks have been received, gracefully handling the following events:
* Broker connection failure
* Topic leader change
* Produce errors signaled by the broker
This is handled automatically by librdkafka and the application does not need
to take any action at any of the above events.
The message will be resent up to "message.send.max.retries" times before
reporting a failure back to the application.
The delivery report callback is used by librdkafka to signal the status of
a message back to the application, it will be called once for each message
to report the status of message delivery:
* If `error_code` is non-zero the message delivery failed and the error_code
indicates the nature of the failure (`rd_kafka_resp_err_t` enum).
* If `error_code` is zero the message has been successfully delivered.
See Producer API chapter for more details on delivery report callback usage.
The delivery report callback is optional.
## Usage
### Documentation
The librdkafka API is documented in the
[`rdkafka.h`](https://github.com/edenhill/librdkafka/blob/master/src/rdkafka.h)
header file, the configuration properties are documented in
[`CONFIGURATION.md`](https://github.com/edenhill/librdkafka/blob/master/CONFIGURATION.md)
### Initialization
The application needs to instantiate a top-level object `rd_kafka_t` which is
the base container, providing global configuration and shared state.
It is created by calling `rd_kafka_new()`.
It also needs to instantiate one or more topics (`rd_kafka_topic_t`) to be used
for producing to or consuming from. The topic object holds topic-specific
configuration and will be internally populated with a mapping of all available
partitions and their leader brokers.
It is created by calling `rd_kafka_topic_new()`.
Both `rd_kafka_t` and `rd_kafka_topic_t` comes with a configuration API which
is optional.
Not using the API will cause librdkafka to use its default values which are
documented *`CONFIGURATION.md`*.
**Note**: An application may create multiple `rd_kafka_t` objects and
they share no state.
**Note**: An `rd_kafka_topic_t` object may only be used with the `rd_kafka_t`
object it was created from.
### Configuration
To ease integration with the official Apache Kafka software and lower
the learning curve, librdkafka implements identical configuration
properties as found in the official clients of Apache Kafka.
Configuration is applied prior to object creation using the
`rd_kafka_conf_set()` and `rd_kafka_topic_conf_set()` APIs.
**Note**: The `rd_kafka.._conf_t` objects are not reusable after have been
passed to `rd_kafka.._new()`.
The application does not need to free any config resources after a
`rd_kafka.._new()` call.
#### Example
rd_kafka_conf_t *conf;
char errstr[512];
conf = rd_kafka_conf_new();
rd_kafka_conf_set(conf, "compression.codec", "snappy", errstr, sizeof(errstr));
rd_kafka_conf_set(conf, "batch.num.messages", "100", errstr, sizeof(errstr));
rd_kafka_new(RD_KAFKA_PRODUCER, conf);
### Threads and callbacks
librdkafka uses multiple threads internally to fully utilize modern hardware.
The API is completely thread-safe and the calling application may call any
of the API functions from any of its own threads at any time.
A poll-based API is used to provide signaling back to the application,
the application should call rd_kafka_poll() at regular intervals.
The poll API will call the following configured callbacks (optional):
* message delivery report callback - signals that a message has been
delivered or failed delivery, allowing the application to take action
and to release any application resources used in the message.
* error callback - signals an error. These errors are usually of an
informational nature, i.e., failure to connect to a broker, and the
application usually does not need to take any action.
The type of error is passed as a rd_kafka_resp_err_t enum value,
including both remote broker errors as well as local failures.
Optional callbacks not triggered by poll, these may be called from any thread:
* Logging callback - allows the application to output log messages
generated by librdkafka.
* partitioner callback - application provided message partitioner.
The partitioner may be called in any thread at any time, it may be
called multiple times for the same key.
Partitioner function contraints:
* MUST NOT call any rd_kafka_*() functions
* MUST NOT block or execute for prolonged periods of time.
* MUST return a value between 0 and partition_cnt-1, or the
special RD_KAFKA_PARTITION_UA value if partitioning
could not be performed.
### Brokers
librdkafka only needs an initial list of brokers (at least one), called the
bootstrap brokers.
It will connect to all the bootstrap brokers, specified by the
"metadata.broker.list" configuration property or by `rd_kafka_brokers_add()`,
and query each one for Metadata information which contains the full list of
brokers, topic, partitions and their leaders in the Kafka cluster.
Broker names are specified as "host[:port]" where the port is optional
(default 9092) and the host is either a resolvable hostname or an IPv4 or IPv6
address.
If host resolves to multiple addresses librdkafka will round-robin the
addresses for each connection attempt.
A DNS record containing all broker address can thus be used to provide a
reliable bootstrap broker.
### Feature discovery
Apache Kafka broker version 0.10.0 added support for the ApiVersionRequest API
which allows a client to query a broker for its range of supported API versions.
librdkafka supports this functionality and will query each broker on connect
for this information (if `api.version.request=true`) and use it to enable or disable
various protocol features, such as MessageVersion 1 (timestamps), KafkaConsumer, etc.
If the broker fails to respond to the ApiVersionRequest librdkafka will
assume the broker is too old to support the API and fall back to an older
broker version's API. These fallback versions are hardcoded in librdkafka
and is controlled by the `broker.version.fallback` configuration property.
### Producer API
After setting up the `rd_kafka_t` object with type `RD_KAFKA_PRODUCER` and one
or more `rd_kafka_topic_t` objects librdkafka is ready for accepting messages
to be produced and sent to brokers.
The `rd_kafka_produce()` function takes the following arguments:
* `rkt` - the topic to produce to, previously created with
`rd_kafka_topic_new()`
* `partition` - partition to produce to. If this is set to
`RD_KAFKA_PARTITION_UA` (UnAssigned) then the configured partitioner
function will be used to select a target partition.
* `msgflags` - 0, or one of:
* `RD_KAFKA_MSG_F_COPY` - librdkafka will immediately make a copy of
the payload. Use this when the payload is in non-persistent
memory, such as the stack.
* `RD_KAFKA_MSG_F_FREE` - let librdkafka free the payload using
`free(3)` when it is done with it.
These two flags are mutually exclusive and neither need to be set in
which case the payload is neither copied nor freed by librdkafka.
If `RD_KAFKA_MSG_F_COPY` flag is not set no data copying will be
performed and librdkafka will hold on the payload pointer until
the message has been delivered or fails.
The delivery report callback will be called when librdkafka is done
with the message to let the application regain ownership of the
payload memory.
The application must not free the payload in the delivery report
callback if `RD_KAFKA_MSG_F_FREE is set`.
* `payload`,`len` - the message payload
* `key`,`keylen` - an optional message key which can be used for partitioning.
It will be passed to the topic partitioner callback, if any, and
will be attached to the message when sending to the broker.
* `msg_opaque` - an optional application-provided per-message opaque pointer
that will be provided in the message delivery callback to let
the application reference a specific message.
`rd_kafka_produce()` is a non-blocking API, it will enqueue the message
on an internal queue and return immediately.
If the number of queued messages would exceed the "queue.buffering.max.messages"
configuration property then `rd_kafka_produce()` returns -1 and sets errno
to `ENOBUFS`, thus providing a backpressure mechanism.
**Note**: See `examples/rdkafka_performance.c` for a producer implementation.
### Simple Consumer API (legacy)
NOTE: For the high-level KafkaConsumer interface see rd_kafka_subscribe (rdkafka.h) or KafkaConsumer (rdkafkacpp.h)
The consumer API is a bit more stateful than the producer API.
After creating `rd_kafka_t` with type `RD_KAFKA_CONSUMER` and
`rd_kafka_topic_t` instances the application must also start the consumer
for a given partition by calling `rd_kafka_consume_start()`.
`rd_kafka_consume_start()` arguments:
* `rkt` - the topic to start consuming from, previously created with
`rd_kafka_topic_new()`.
* `partition` - partition to consume from.
* `offset` - message offset to start consuming from. This may either be an
absolute message offset or one of the two special offsets:
`RD_KAFKA_OFFSET_BEGINNING` to start consuming from the beginning
of the partition's queue (oldest message), or
`RD_KAFKA_OFFSET_END` to start consuming at the next message to be
produced to the partition, or
`RD_KAFKA_OFFSET_STORED` to use the offset store.
After a topic+partition consumer has been started librdkafka will attempt
to keep "queued.min.messages" messages in the local queue by repeatedly
fetching batches of messages from the broker.
This local message queue is then served to the application through three
different consume APIs:
* `rd_kafka_consume()` - consumes a single message
* `rd_kafka_consume_batch()` - consumes one or more messages
* `rd_kafka_consume_callback()` - consumes all messages in the local
queue and calls a callback function for each one.
These three APIs are listed above the ascending order of performance,
`rd_kafka_consume()` being the slowest and `rd_kafka_consume_callback()` being
the fastest. The different consume variants are provided to cater for different
application needs.
A consumed message, as provided or returned by each of the consume functions,
is represented by the `rd_kafka_message_t` type.
`rd_kafka_message_t` members:
* `err` - Error signaling back to the application. If this field is non-zero
the `payload` field should be considered an error message and
`err` is an error code (`rd_kafka_resp_err_t`).
If `err` is zero then the message is a proper fetched message
and `payload` et.al contains message payload data.
* `rkt`,`partition` - Topic and partition for this message or error.
* `payload`,`len` - Message payload data or error message (err!=0).
* `key`,`key_len` - Optional message key as specified by the producer
* `offset` - Message offset
Both the `payload` and `key` memory, as well as the message as a whole, is
owned by librdkafka and must not be used after an `rd_kafka_message_destroy()`
call. librdkafka will share the same messageset receive buffer memory for all
message payloads of that messageset to avoid excessive copying which means
that if the application decides to hang on to a single `rd_kafka_message_t`
it will hinder the backing memory to be released for all other messages
from the same messageset.
When the application is done consuming messages from a topic+partition it
should call `rd_kafka_consume_stop()` to stop the consumer. This will also
purge any messages currently in the local queue.
**Note**: See `examples/rdkafka_performance.c` for a consumer implementation.
#### Offset management
Broker based offset management is available for broker version >= 0.9.0
in conjunction with using the high-level KafkaConsumer interface (see
rdkafka.h or rdkafkacpp.h)
Offset management is also available through a local offset file store, where the
offset is periodically written to a local file for each topic+partition
according to the following topic configuration properties:
* `auto.commit.enable`
* `auto.commit.interval.ms`
* `offset.store.path`
* `offset.store.sync.interval.ms`
There is currently no support for offset management with ZooKeeper.
#### Consumer groups
Broker based consumer groups (requires Apache Kafka broker >=0.9) are supported,
see KafkaConsumer in rdkafka.h or rdkafkacpp.h
### Topics
#### Topic auto creation
Topic auto creation is supported by librdkafka.
The broker needs to be configured with "auto.create.topics.enable=true".
## Appendix
### Test details
#### Test1: Produce to two brokers, two partitions, required.acks=2, 100 byte messages
Each broker is leader for one of the two partitions.
The random partitioner is used (default) and each broker and partition is
assigned approximately 250000 messages each.
**Command:**
# examples/rdkafka_performance -P -t test2 -s 100 -c 500000 -m "_____________Test1:TwoBrokers:500kmsgs:100bytes" -S 1 -a 2
....
% 500000 messages and 50000000 bytes sent in 587ms: 851531 msgs/s and 85.15 Mb/s, 0 messages failed, no compression
**Result:**
Message transfer rate is approximately **850000 messages per second**,
**85 megabytes per second**.
#### Test2: Produce to one broker, one partition, required.acks=0, 100 byte messages
**Command:**
# examples/rdkafka_performance -P -t test2 -s 100 -c 500000 -m "_____________Test2:OneBrokers:500kmsgs:100bytes" -S 1 -a 0 -p 1
....
% 500000 messages and 50000000 bytes sent in 698ms: 715994 msgs/s and 71.60 Mb/s, 0 messages failed, no compression
**Result:**
Message transfer rate is approximately **710000 messages per second**,
**71 megabytes per second**.
#### Test3: Produce to two brokers, two partitions, required.acks=2, 100 byte messages, snappy compression
**Command:**
# examples/rdkafka_performance -P -t test2 -s 100 -c 500000 -m "_____________Test3:TwoBrokers:500kmsgs:100bytes:snappy" -S 1 -a 2 -z snappy
....
% 500000 messages and 50000000 bytes sent in 1672ms: 298915 msgs/s and 29.89 Mb/s, 0 messages failed, snappy compression
**Result:**
Message transfer rate is approximately **300000 messages per second**,
**30 megabytes per second**.
#### Test4: Produce to two brokers, two partitions, required.acks=2, 100 byte messages, gzip compression
**Command:**
# examples/rdkafka_performance -P -t test2 -s 100 -c 500000 -m "_____________Test3:TwoBrokers:500kmsgs:100bytes:gzip" -S 1 -a 2 -z gzip
....
% 500000 messages and 50000000 bytes sent in 2111ms: 236812 msgs/s and 23.68 Mb/s, 0 messages failed, gzip compression
**Result:**
Message transfer rate is approximately **230000 messages per second**,
**23 megabytes per second**.
librdkafka-0.9.3/LICENCES.rtf 0000664 0000000 0000000 00000027537 13041406524 0015534 0 ustar 00root root 0000000 0000000 {\rtf1\ansi\ansicpg1252\deff0{\fonttbl{\f0\fnil\fcharset0 Courier New;}{\f1\fnil\fcharset0 Calibri;}}
{\colortbl ;\red0\green0\blue255;}
{\*\generator Msftedit 5.41.21.2510;}\viewkind4\uc1\pard\lang1053\f0\fs22 librdkafka - Apache Kafka C driver library\par
\par
Copyright (c) 2012-2015, Magnus Edenhill\par
All rights reserved.\par
\par
Redistribution and use in source and binary forms, with or without\par
modification, are permitted provided that the following conditions are met: \par
\par
1. Redistributions of source code must retain the above copyright notice,\par
this list of conditions and the following disclaimer. \par
2. Redistributions in binary form must reproduce the above copyright notice,\par
this list of conditions and the following disclaimer in the documentation\par
and/or other materials provided with the distribution. \par
\par
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"\par
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE \par
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE \par
ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE \par
LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR \par
CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF \par
SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS \par
INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN \par
CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)\par
ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE\par
POSSIBILITY OF SUCH DAMAGE.\par
\par
\par
\par
#######################################################################\par
# The following license applies to the files rdcrc32.c and rdcrc32.h #\par
# which have been generated by the pycrc tool. #\par
#######################################################################\par
\par
Copyright (c) 2006-2012, Thomas Pircher \par
\par
Permission is hereby granted, free of charge, to any person obtaining a copy\par
of this software and associated documentation files (the "Software"), to deal\par
in the Software without restriction, including without limitation the rights\par
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell\par
copies of the Software, and to permit persons to whom the Software is\par
furnished to do so, subject to the following conditions:\par
\par
The above copyright notice and this permission notice shall be included in\par
all copies or substantial portions of the Software.\par
\par
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\par
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\par
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\par
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\par
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\par
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN\par
THE SOFTWARE.\par
\pard\sa200\sl276\slmult1\lang29\f1\par
\par
\pard\lang1053\f0 ######################################################################\par
# LICENSE.snappy covers files: snappy.c, snappy.h, snappy_compat.h #\par
# originally retrieved from {\field{\*\fldinst{HYPERLINK "http://github.com/andikleen/snappy-c"}}{\fldrslt{\ul\cf1 http://github.com/andikleen/snappy-c}}}\f0\fs22 #\par
# git revision 711c52b7ef94c8e5c600571987fbe5769070b884\tab #\par
######################################################################\par
\par
The snappy-c code is under the same license as the original snappy source\par
\par
Copyright 2011 Intel Corporation All Rights Reserved.\par
\par
Redistribution and use in source and binary forms, with or without\par
modification, are permitted provided that the following conditions are\par
met:\par
\par
* Redistributions of source code must retain the above copyright\par
notice, this list of conditions and the following disclaimer.\par
* Redistributions in binary form must reproduce the above\par
copyright notice, this list of conditions and the following disclaimer\par
in the documentation and/or other materials provided with the\par
distribution.\par
* Neither the name of Intel Corporation nor the names of its\par
contributors may be used to endorse or promote products derived from\par
this software without specific prior written permission.\par
\par
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS\par
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT\par
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR\par
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT\par
OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,\par
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT\par
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,\par
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY\par
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT\par
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE\par
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.\par
\par
\par
######################################################################\par
# tinycthread.c and tinycthread.h #\par
######################################################################\par
From {\field{\*\fldinst{HYPERLINK "https://github.com/tinycthread/tinycthread/README.txt"}}{\fldrslt{\ul\cf1 https://github.com/tinycthread/tinycthread/README.txt}}}\f0\fs22 c57166cd510ffb5022dd5f127489b131b61441b9\par
\par
License\par
-------\par
\par
Copyright (c) 2012 Marcus Geelnard\par
2013-2014 Evan Nemerson\par
\par
This software is provided 'as-is', without any express or implied\par
warranty. In no event will the authors be held liable for any damages\par
arising from the use of this software.\par
\par
Permission is granted to anyone to use this software for any purpose,\par
including commercial applications, and to alter it and redistribute it\par
freely, subject to the following restrictions:\par
\par
1. The origin of this software must not be misrepresented; you must not\par
claim that you wrote the original software. If you use this software\par
in a product, an acknowledgment in the product documentation would be\par
appreciated but is not required.\par
\par
2. Altered source versions must be plainly marked as such, and must not be\par
misrepresented as being the original software.\par
\par
3. This notice may not be removed or altered from any source\par
distribution.\par
\par
\par
\par
################################################################\par
# For the files wingetopt.c wingetopt.h downloaded from\par
# {\field{\*\fldinst{HYPERLINK "https://github.com/alex85k/wingetopt"}}{\fldrslt{\ul\cf1 https://github.com/alex85k/wingetopt}}}\f0\fs22\par
################################################################\par
\par
\par
/*\par
* Copyright (c) 2002 Todd C. Miller \par
*\par
* Permission to use, copy, modify, and distribute this software for any\par
* purpose with or without fee is hereby granted, provided that the above\par
* copyright notice and this permission notice appear in all copies.\par
*\par
* THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES\par
* WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF\par
* MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR\par
* ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES\par
* WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN\par
* ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF\par
* OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.\par
*\par
* Sponsored in part by the Defense Advanced Research Projects\par
* Agency (DARPA) and Air Force Research Laboratory, Air Force\par
* Materiel Command, USAF, under agreement number F39502-99-1-0512.\par
*/\par
/*-\par
* Copyright (c) 2000 The NetBSD Foundation, Inc.\par
* All rights reserved.\par
*\par
* This code is derived from software contributed to The NetBSD Foundation\par
* by Dieter Baron and Thomas Klausner.\par
*\par
* Redistribution and use in source and binary forms, with or without\par
* modification, are permitted provided that the following conditions\par
* are met:\par
* 1. Redistributions of source code must retain the above copyright\par
* notice, this list of conditions and the following disclaimer.\par
* 2. Redistributions in binary form must reproduce the above copyright\par
* notice, this list of conditions and the following disclaimer in the\par
* documentation and/or other materials provided with the distribution.\par
*\par
* THIS SOFTWARE IS PROVIDED BY THE NETBSD FOUNDATION, INC. AND CONTRIBUTORS\par
* ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED\par
* TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR\par
* PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE FOUNDATION OR CONTRIBUTORS\par
* BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR\par
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF\par
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS\par
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN\par
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)\par
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE\par
* POSSIBILITY OF SUCH DAMAGE.\par
*/\par
\par
\par
########################################################################\par
# src/queue.h\par
########################################################################\par
For sys/queue.h:\par
\par
* Copyright (c) 1991, 1993\par
*\tab The Regents of the University of California. All rights reserved.\par
*\par
* Redistribution and use in source and binary forms, with or without\par
* modification, are permitted provided that the following conditions\par
* are met:\par
* 1. Redistributions of source code must retain the above copyright\par
* notice, this list of conditions and the following disclaimer.\par
* 2. Redistributions in binary form must reproduce the above copyright\par
* notice, this list of conditions and the following disclaimer in the\par
* documentation and/or other materials provided with the distribution.\par
* 4. Neither the name of the University nor the names of its contributors\par
* may be used to endorse or promote products derived from this software\par
* without specific prior written permission.\par
*\par
* THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND\par
* ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE\par
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE\par
* ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE\par
* FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL\par
* DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS\par
* OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)\par
* HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT\par
* LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY\par
* OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF\par
* SUCH DAMAGE.\par
*\par
*\tab @(#)queue.h\tab 8.5 (Berkeley) 8/20/94\par
* $FreeBSD$\par
\par
\par
}
librdkafka-0.9.3/LICENSE 0000664 0000000 0000000 00000002513 13041406524 0014622 0 ustar 00root root 0000000 0000000 librdkafka - Apache Kafka C driver library
Copyright (c) 2012, Magnus Edenhill
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
1. Redistributions of source code must retain the above copyright notice,
this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
POSSIBILITY OF SUCH DAMAGE.
librdkafka-0.9.3/LICENSE.pycrc 0000664 0000000 0000000 00000002367 13041406524 0015750 0 ustar 00root root 0000000 0000000 The following license applies to the files rdcrc32.c and rdcrc32.h which
have been generated by the pycrc tool.
============================================================================
Copyright (c) 2006-2012, Thomas Pircher
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
librdkafka-0.9.3/LICENSE.queue 0000664 0000000 0000000 00000003172 13041406524 0015747 0 ustar 00root root 0000000 0000000 For sys/queue.h:
* Copyright (c) 1991, 1993
* The Regents of the University of California. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* 4. Neither the name of the University nor the names of its contributors
* may be used to endorse or promote products derived from this software
* without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND
* ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE
* FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
* DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
* OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
* HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
* LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
* OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
* SUCH DAMAGE.
*
* @(#)queue.h 8.5 (Berkeley) 8/20/94
* $FreeBSD$ librdkafka-0.9.3/LICENSE.regexp 0000664 0000000 0000000 00000000506 13041406524 0016113 0 ustar 00root root 0000000 0000000 regexp.c and regexp.h from https://github.com/ccxvii/minilibs sha 875c33568b5a4aa4fb3dd0c52ea98f7f0e5ca684
"
These libraries are in the public domain (or the equivalent where that is not possible). You can do anything you want with them. You have no legal obligation to do anything else, although I appreciate attribution.
"
librdkafka-0.9.3/LICENSE.snappy 0000664 0000000 0000000 00000003576 13041406524 0016145 0 ustar 00root root 0000000 0000000 ######################################################################
# LICENSE.snappy covers files: snappy.c, snappy.h, snappy_compat.h #
# originally retrieved from http://github.com/andikleen/snappy-c #
# git revision 8015f2d28739b9a6076ebaa6c53fe27bc238d219 #
######################################################################
The snappy-c code is under the same license as the original snappy source
Copyright 2011 Intel Corporation All Rights Reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
met:
* Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above
copyright notice, this list of conditions and the following disclaimer
in the documentation and/or other materials provided with the
distribution.
* Neither the name of Intel Corporation nor the names of its
contributors may be used to endorse or promote products derived from
this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
librdkafka-0.9.3/LICENSE.tinycthread 0000664 0000000 0000000 00000002010 13041406524 0017127 0 ustar 00root root 0000000 0000000 From https://github.com/tinycthread/tinycthread/README.txt c57166cd510ffb5022dd5f127489b131b61441b9
License
-------
Copyright (c) 2012 Marcus Geelnard
2013-2014 Evan Nemerson
This software is provided 'as-is', without any express or implied
warranty. In no event will the authors be held liable for any damages
arising from the use of this software.
Permission is granted to anyone to use this software for any purpose,
including commercial applications, and to alter it and redistribute it
freely, subject to the following restrictions:
1. The origin of this software must not be misrepresented; you must not
claim that you wrote the original software. If you use this software
in a product, an acknowledgment in the product documentation would be
appreciated but is not required.
2. Altered source versions must be plainly marked as such, and must not be
misrepresented as being the original software.
3. This notice may not be removed or altered from any source
distribution.
librdkafka-0.9.3/LICENSE.wingetopt 0000664 0000000 0000000 00000005046 13041406524 0016645 0 ustar 00root root 0000000 0000000 For the files wingetopt.c wingetopt.h downloaded from https://github.com/alex85k/wingetopt
/*
* Copyright (c) 2002 Todd C. Miller
*
* Permission to use, copy, modify, and distribute this software for any
* purpose with or without fee is hereby granted, provided that the above
* copyright notice and this permission notice appear in all copies.
*
* THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
* WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
* MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
* ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
* WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
* ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
* OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*
* Sponsored in part by the Defense Advanced Research Projects
* Agency (DARPA) and Air Force Research Laboratory, Air Force
* Materiel Command, USAF, under agreement number F39502-99-1-0512.
*/
/*-
* Copyright (c) 2000 The NetBSD Foundation, Inc.
* All rights reserved.
*
* This code is derived from software contributed to The NetBSD Foundation
* by Dieter Baron and Thomas Klausner.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
*
* THIS SOFTWARE IS PROVIDED BY THE NETBSD FOUNDATION, INC. AND CONTRIBUTORS
* ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
* TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
* PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE FOUNDATION OR CONTRIBUTORS
* BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*/
librdkafka-0.9.3/LICENSE.xxhash 0000664 0000000 0000000 00000002600 13041406524 0016121 0 ustar 00root root 0000000 0000000 src/xxhash.[ch]: git@github.com:Cyan4973/xxHash.git fec956e55ccab2a80779ab96fce38f85341fa861
xxHash Library
Copyright (c) 2012-2014, Yann Collet
All rights reserved.
Redistribution and use in source and binary forms, with or without modification,
are permitted provided that the following conditions are met:
* Redistributions of source code must retain the above copyright notice, this
list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright notice, this
list of conditions and the following disclaimer in the documentation and/or
other materials provided with the distribution.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR
ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
librdkafka-0.9.3/Makefile 0000775 0000000 0000000 00000003003 13041406524 0015253 0 ustar 00root root 0000000 0000000 LIBSUBDIRS= src src-cpp
CHECK_FILES+= CONFIGURATION.md \
examples/rdkafka_example examples/rdkafka_performance \
examples/rdkafka_example_cpp
PACKAGE_NAME?= librdkafka
VERSION?= $(shell python packaging/get_version.py src/rdkafka.h)
# Jenkins CI integration
BUILD_NUMBER ?= 1
.PHONY:
all: mklove-check libs CONFIGURATION.md check
include mklove/Makefile.base
libs:
@(for d in $(LIBSUBDIRS); do $(MAKE) -C $$d || exit $?; done)
CONFIGURATION.md: src/rdkafka.h examples
@printf "$(MKL_YELLOW)Updating$(MKL_CLR_RESET)\n"
@echo '//@file' > CONFIGURATION.md.tmp
@(examples/rdkafka_performance -X list >> CONFIGURATION.md.tmp; \
cmp CONFIGURATION.md CONFIGURATION.md.tmp || \
mv CONFIGURATION.md.tmp CONFIGURATION.md; \
rm -f CONFIGURATION.md.tmp)
file-check: CONFIGURATION.md examples
check: file-check
@(for d in $(LIBSUBDIRS); do $(MAKE) -C $$d $@ || exit $?; done)
install:
@(for d in $(LIBSUBDIRS); do $(MAKE) -C $$d $@ || exit $?; done)
examples tests: .PHONY libs
$(MAKE) -C $@
docs:
doxygen Doxyfile
@echo "Documentation generated in staging-docs"
clean-docs:
rm -rf staging-docs
clean:
@$(MAKE) -C tests $@
@$(MAKE) -C examples $@
@(for d in $(LIBSUBDIRS); do $(MAKE) -C $$d $@ ; done)
distclean: clean
./configure --clean
rm -f config.log config.log.old
archive:
git archive --prefix=$(PACKAGE_NAME)-$(VERSION)/ \
-o $(PACKAGE_NAME)-$(VERSION).tar.gz HEAD
git archive --prefix=$(PACKAGE_NAME)-$(VERSION)/ \
-o $(PACKAGE_NAME)-$(VERSION).zip HEAD
rpm: distclean
$(MAKE) -C packaging/rpm
librdkafka-0.9.3/README.md 0000664 0000000 0000000 00000013454 13041406524 0015102 0 ustar 00root root 0000000 0000000 librdkafka - Apache Kafka C/C++ client library
==============================================
Copyright (c) 2012-2016, [Magnus Edenhill](http://www.edenhill.se/).
[https://github.com/edenhill/librdkafka](https://github.com/edenhill/librdkafka)
**librdkafka** is a C library implementation of the
[Apache Kafka](http://kafka.apache.org/) protocol, containing both
Producer and Consumer support. It was designed with message delivery reliability
and high performance in mind, current figures exceed 1 million msgs/second for
the producer and 3 million msgs/second for the consumer.
**librdkafka** is licensed under the 2-clause BSD license.
For an introduction to the performance and usage of librdkafka, see
[INTRODUCTION.md](https://github.com/edenhill/librdkafka/blob/master/INTRODUCTION.md)
See the [wiki](https://github.com/edenhill/librdkafka/wiki) for a FAQ.
**NOTE**: The `master` branch is actively developed, use latest release for production use.
[](https://gitter.im/edenhill/librdkafka)
**Apache Kafka 0.8 & 0.9 & 0.10 support:**
* Branch: master
* Producer: supported
* High-level balanced KafkaConsumer: supported (requires broker >= 0.9)
* Simple Consumer: supported
* Compression: snappy, gzip, lz4
* Broker version support: >=0.8
* Debian package: librdkafka1 and librdkafka-dev in Debian and Ubuntu
* ZooKeeper: not supported
* C API: Stable, ABI safe
* C++ API: Stable
* Tests: Regression tests in `tests/` directory.
* Statistics: JSON formatted, see `rd_kafka_conf_set_stats_cb` in `rdkafka.h`.
* Status: Testing
**Apache Kafka 0.7 support:**
* Branch: 0.7
* Producer: supported
* High-level Consumer: not supported
* Simple Consumer: supported
* Compression: not supported
* Broker version support: 0.7.x
* ZooKeeper: not supported
* Status: Stable, Deprecated, Unsupported
#Language bindings#
* Haskell: [haskakafka](https://github.com/cosbynator/haskakafka)
* Haskell: [haskell-kafka](https://github.com/yanatan16/haskell-kafka)
* Node.js: [node-kafka](https://github.com/sutoiku/node-kafka)
* Node.js: [kafka-native](https://github.com/jut-io/node-kafka-native)
* Lua: [luardkafka](https://github.com/mistsv/luardkafka)
* OCaml: [ocaml-kafka](https://github.com/didier-wenzek/ocaml-kafka)
* PHP: [phpkafka](https://github.com/EVODelavega/phpkafka)
* PHP: [php-rdkafka](https://github.com/arnaud-lb/php-rdkafka)
* Python: [confluent-kafka-python](https://github.com/confluentinc/confluent-kafka-python)
* Python: [PyKafka](https://github.com/Parsely/pykafka)
* Ruby: [Hermann](https://github.com/stancampbell3/Hermann)
* Tcl: [KafkaTcl](https://github.com/flightaware/kafkatcl)
* C#/.NET: [rdkafka-dotnet](https://github.com/ah-/rdkafka-dotnet)
* D (C-like): [librdkafka](https://github.com/DlangApache/librdkafka/)
* D (C++-like): [librdkafkad](https://github.com/tamediadigital/librdkafka-d)
#Users of librdkafka#
* [kafkacat](https://github.com/edenhill/kafkacat) - Apache Kafka swiss army knife
* [Wikimedia's varnishkafka](https://github.com/wikimedia/varnishkafka) - Varnish cache web log producer
* [Wikimedia's kafkatee](https://github.com/wikimedia/analytics-kafkatee) - Kafka multi consumer with filtering and fanout
* [rsyslog](http://www.rsyslog.com)
* [syslog-ng](http://syslog-ng.org)
* [collectd](http://collectd.org)
* [logkafka](https://github.com/Qihoo360/logkafka) - Collect logs and send to Kafka
* [redBorder](http://www.redborder.net)
* [Headweb](http://www.headweb.com/)
* [Produban's log2kafka](https://github.com/Produban/log2kafka) - Web log producer
* [fuse_kafka](https://github.com/yazgoo/fuse_kafka) - FUSE file system layer
* [node-kafkacat](https://github.com/Rafflecopter/node-kafkacat)
* [OVH](http://ovh.com) - [AntiDDOS](http://www.slideshare.net/hugfrance/hugfr-6-oct2014ovhantiddos)
* [otto.de](http://otto.de)'s [trackdrd](https://github.com/otto-de/trackrdrd) - Varnish log reader
* [Microwish](https://github.com/microwish) has a range of Kafka utilites for log aggregation, HDFS integration, etc.
* large unnamed financial institution
* *Let [me](mailto:rdkafka@edenhill.se) know if you are using librdkafka*
# Usage
## Requirements
The GNU toolchain
GNU make
pthreads
zlib (optional, for gzip compression support)
libssl-dev (optional, for SSL support)
libsasl2-dev (optional, for SASL support)
liblz4-dev (optional, for LZ4 compression support)
## Instructions
### Building
./configure
make
sudo make install
**NOTE**: See [README.win32](README.win32) for instructions how to build
on Windows with Microsoft Visual Studio.
### Usage in code
See [examples/rdkafka_example.c](https://github.com/edenhill/librdkafka/blob/master/examples/rdkafka_example.c) for an example producer and consumer.
Link your program with `-lrdkafka -lz -lpthread -lrt`.
## Documentation
The public APIs are documented in their respective header files:
* The **C** API is documented in [src/rdkafka.h](src/rdkafka.h)
* The **C++** API is documented in [src-cpp/rdkafkacpp.h](src-cpp/rdkafkacpp.h)
To generate Doxygen documents for the API, type:
make docs
Configuration properties are documented in
[CONFIGURATION.md](https://github.com/edenhill/librdkafka/blob/master/CONFIGURATION.md)
For a librdkafka introduction, see
[INTRODUCTION.md](https://github.com/edenhill/librdkafka/blob/master/INTRODUCTION.md)
## Examples
See the `examples/`sub-directory.
## Tests
See the `tests/`sub-directory.
## Support
File bug reports, feature requests and questions using
[GitHub Issues](https://github.com/edenhill/librdkafka/issues)
Questions and discussions are also welcome on irc.freenode.org, #apache-kafka,
nickname Snaps.
### Commercial support
Commercial support is available from [Edenhill services](http://www.edenhill.se)
librdkafka-0.9.3/README.win32 0000664 0000000 0000000 00000001710 13041406524 0015434 0 ustar 00root root 0000000 0000000
Native win32 build instructions using Microsoft Visual Studio 2013 (MSVC).
Requirements:
* zlib is installed automatically from NuGet,
but probably requires the NuGet VS extension.
* OpenSSL-win32 must be installed in C:\OpenSSL-win32.
Download and install the latest v1.0.2 non-light package from:
https://slproweb.com/products/Win32OpenSSL.html
(This would be using NuGet too but the current
OpenSSL packages are outdated and with broken
dependencies, so no luck)
The Visual Studio solution file for librdkafka resides in win32/librdkafka.sln
Artifacts:
- C library
- C++ library
- rdkafka_example
- tests
Missing:
- remaining tools (rdkafka_performance, etc)
- SASL support (no official Cyrus libsasl2 DLLs available)
- LZ4 support
If you build librdkafka with an external tool (ie CMake) you can get rid of the
__declspec(dllexport) / __declspec(dllimport) decorations by adding a define
-DLIBRDKAFKA_STATICLIB to your CFLAGS
librdkafka-0.9.3/configure 0000775 0000000 0000000 00000011040 13041406524 0015517 0 ustar 00root root 0000000 0000000 #!/usr/bin/env bash
#
BASHVER=$(expr ${BASH_VERSINFO[0]} \* 1000 + ${BASH_VERSINFO[1]})
if [ "$BASHVER" -lt 3002 ]; then
echo "ERROR: mklove requires bash version 3.2 or later but you are using $BASH_VERSION ($BASHVER)"
echo " See https://github.com/edenhill/mklove/issues/15"
exit 1
fi
MKL_CONFIGURE_ARGS="$0 $*"
# Load base module
source mklove/modules/configure.base
# Read some special command line options right away that must be known prior to
# sourcing modules.
mkl_in_list "$*" "--no-download" && MKL_NO_DOWNLOAD=1
# Disable downloads when --help is used to avoid blocking calls.
mkl_in_list "$*" "--help" && MKL_NO_DOWNLOAD=1
mkl_in_list "$*" "--debug" && MKL_DEBUG=1
# This is the earliest possible time to check for color support in
# terminal because mkl_check_terminal_color_support uses mkl_dbg which
# needs to know if MKL_DEBUG is set
mkl_check_terminal_color_support
# Delete temporary Makefile and header files on exit.
trap "{ rm -f $MKL_OUTMK $MKL_OUTH; }" EXIT
##
## Load builtin modules
##
# Builtin options, etc.
mkl_require builtin
# Host/target support
mkl_require host
# Compiler detection
mkl_require cc
# Load application provided modules (in current directory), if any.
for fname in configure.* ; do
if [[ $fname = 'configure.*' ]]; then
continue
fi
# Skip temporary files
if [[ $fname = *~ ]]; then
continue
fi
mkl_require $fname
done
##
## Argument parsing (options)
##
##
_SAVE_ARGS="$*"
# Parse arguments
while [[ ! -z $@ ]]; do
if [[ $1 != --* ]]; then
mkl_err "Unknown non-option argument: $1"
mkl_usage
exit 1
fi
opt=${1#--}
shift
if [[ $opt = *=* ]]; then
name="${opt%=*}"
arg="${opt#*=}"
eqarg=1
else
name="$opt"
arg=""
eqarg=0
fi
safeopt="$(mkl_env_esc $name)"
if ! mkl_func_exists opt_$safeopt ; then
mkl_err "Unknown option $opt"
mkl_usage
exit 1
fi
# Check if this option needs an argument.
reqarg=$(mkl_meta_get "MKL_OPT_ARGS" "$(mkl_env_esc $name)")
if [[ ! -z $reqarg ]]; then
if [[ $eqarg == 0 && -z $arg ]]; then
arg=$1
shift
if [[ -z $arg ]]; then
mkl_err "Missing argument to option --$name $reqarg"
exit 1
fi
fi
else
if [[ ! -z $arg ]]; then
mkl_err "Option --$name expects no argument"
exit 1
fi
arg=y
fi
case $name in
re|reconfigure)
oldcmd=$(head -1 config.log | grep '^# configure exec: ' | \
sed -e 's/^\# configure exec: [^ ]*configure//')
echo "Reconfiguring: $0 $oldcmd"
exec $0 $oldcmd
;;
list-modules)
echo "Modules loaded:"
for mod in $MKL_MODULES ; do
echo " $mod"
done
exit 0
;;
list-checks)
echo "Check functions in calling order:"
for mf in $MKL_CHECKS ; do
mod=${mf%:*}
func=${mf#*:}
echo -e "${MKL_GREEN}From module $mod:$MKL_CLR_RESET"
declare -f $func
echo ""
done
exit 0
;;
update-modules)
fails=0
echo "Updating modules"
for mod in $MKL_MODULES ; do
echo -n "Updating $mod..."
if mkl_module_download "$mod" > /dev/null ; then
echo -e "${MKL_GREEN}ok${MKL_CLR_RESET}"
else
echo -e "${MKL_RED}failed${MKL_CLR_RESET}"
fails=$(expr $fails + 1)
fi
done
exit $fails
;;
help)
mkl_usage
exit 0
;;
*)
opt_$safeopt $arg || exit 1
mkl_var_append MKL_OPTS_SET "$safeopt"
;;
esac
done
if [[ ! -z $MKL_CLEAN ]]; then
mkl_clean
exit 0
fi
# Move away previous log file
[[ -f $MKL_OUTDBG ]] && mv $MKL_OUTDBG ${MKL_OUTDBG}.old
# Create output files
echo "# configure exec: $0 $_SAVE_ARGS" >> $MKL_OUTDBG
echo "# On $(date)" >> $MKL_OUTDBG
rm -f $MKL_OUTMK $MKL_OUTH
# Load cache file
mkl_cache_read
# Run checks
mkl_checks_run
# Check accumulated failures, will not return on failure.
mkl_check_fails
# Generate outputs
mkl_generate
# Summarize what happened
mkl_summary
# Write cache file
mkl_cache_write
echo ""
echo "Now type 'make' to build"
trap - EXIT
exit 0
librdkafka-0.9.3/configure.librdkafka 0000664 0000000 0000000 00000010365 13041406524 0017616 0 ustar 00root root 0000000 0000000 #!/bin/bash
#
mkl_meta_set "description" "name" "librdkafka"
mkl_meta_set "description" "oneline" "The Apache Kafka C/C++ library"
mkl_meta_set "description" "long" "Full Apache Kafka protocol support, including producer and consumer"
mkl_meta_set "description" "copyright" "Copyright (c) 2012-2015 Magnus Edenhill"
# Enable generation of pkg-config .pc file
mkl_mkvar_set "" GEN_PKG_CONFIG y
mkl_require cxx
mkl_require lib
mkl_require pic
mkl_require atomics
mkl_require good_cflags
mkl_require socket
# Generate version variables from rdkafka.h hex version define
# so we can use it as string version when generating a pkg-config file.
verdef=$(grep '^#define *RD_KAFKA_VERSION *0x' src/rdkafka.h | sed 's/^#define *RD_KAFKA_VERSION *\(0x[a-f0-9]*\)\.*$/\1/')
mkl_require parseversion hex2str "%d.%d.%d" "$verdef" RDKAFKA_VERSION_STR
mkl_toggle_option "Development" ENABLE_DEVEL "--enable-devel" "Enable development asserts, checks, etc" "n"
mkl_toggle_option "Development" ENABLE_VALGRIND "--enable-valgrind" "Enable in-code valgrind suppressions" "n"
mkl_toggle_option "Development" ENABLE_REFCNT_DEBUG "--enable-refcnt-debug" "Enable refcnt debugging" "n"
mkl_toggle_option "Development" ENABLE_SHAREDPTR_DEBUG "--enable-sharedptr-debug" "Enable sharedptr debugging" "n"
mkl_toggle_option "Feature" ENABLE_SSL "--enable-ssl" "Enable SSL support" "y"
mkl_toggle_option "Feature" ENABLE_SASL "--enable-sasl" "Enable SASL support" "y"
function checks {
# required libs
mkl_lib_check "libpthread" "" fail CC "-lpthread" \
"#include "
# optional libs
mkl_lib_check "zlib" "WITH_ZLIB" disable CC "-lz" \
"#include "
mkl_lib_check "libcrypto" "" disable CC "-lcrypto"
mkl_lib_check "liblz4" "WITH_LZ4" disable CC "-llz4" \
"#include "
# Snappy support is built-in
mkl_allvar_set WITH_SNAPPY WITH_SNAPPY y
# Enable sockem (tests)
mkl_allvar_set WITH_SOCKEM WITH_SOCKEM y
if [[ "$ENABLE_SSL" == "y" ]]; then
mkl_meta_set "libssl" "deb" "libssl-dev"
mkl_lib_check "libssl" "WITH_SSL" disable CC "-lssl" \
"#include "
fi
if [[ "$ENABLE_SASL" == "y" ]]; then
mkl_meta_set "libsasl2" "deb" "libsasl2-dev"
if ! mkl_lib_check "libsasl2" "WITH_SASL" disable CC "-lsasl2" "#include " ; then
mkl_lib_check "libsasl" "WITH_SASL" disable CC "-lsasl" \
"#include "
fi
fi
# Check for libc regex
mkl_compile_check "regex" "HAVE_REGEX" disable CC "" \
"
#include
#include
void foo (void) {
regcomp(NULL, NULL, 0);
regexec(NULL, NULL, 0, NULL, 0);
regerror(0, NULL, NULL, 0);
regfree(NULL);
}"
# -lrt is needed on linux for clock_gettime: link it if it exists.
mkl_lib_check "librt" "" cont CC "-lrt"
# Older g++ (<=4.1?) gives invalid warnings for the C++ code.
mkl_mkvar_append CXXFLAGS CXXFLAGS "-Wno-non-virtual-dtor"
# Required on SunOS
if [[ $MKL_DISTRO == "SunOS" ]]; then
mkl_mkvar_append CPPFLAGS CPPFLAGS "-D_POSIX_PTHREAD_SEMANTICS -D_REENTRANT -D__EXTENSIONS__"
fi
# Check if strndup() is available (isn't on Solaris 10)
mkl_compile_check "strndup" "HAVE_STRNDUP" disable CC "" \
"#include
int foo (void) {
return strndup(\"hi\", 2) ? 0 : 1;
}"
# Figure out what tool to use for dumping public symbols.
# We rely on configure.cc setting up $NM if it exists.
if mkl_env_check "nm" "" cont "NM" ; then
# nm by future mk var
if [[ $MKL_DISTRO == "osx" || $MKL_DISTRO == "AIX" ]]; then
mkl_mkvar_set SYMDUMPER SYMDUMPER '$(NM) -g'
else
mkl_mkvar_set SYMDUMPER SYMDUMPER '$(NM) -D'
fi
else
# Fake symdumper
mkl_mkvar_set SYMDUMPER SYMDUMPER 'echo'
fi
# The linker-script generator (lds-gen.py) requires python
if [[ $WITH_LDS == y ]]; then
if ! mkl_command_check python "HAVE_PYTHON" "disable" "python -V"; then
mkl_err "disabling linker-script since python is not available"
mkl_mkvar_set WITH_LDS WITH_LDS "n"
fi
fi
if [[ "$ENABLE_VALGRIND" == "y" ]]; then
mkl_compile_check valgrind WITH_VALGRIND disable CC "" \
"#include "
fi
}
librdkafka-0.9.3/dev-conf.sh 0000775 0000000 0000000 00000000445 13041406524 0015657 0 ustar 00root root 0000000 0000000 #!/bin/bash
#
# Configure librdkafka for development
set -e
./configure --clean
#export CFLAGS='-std=c99 -pedantic -Wshadow'
#export CXXFLAGS='-std=c++98 -pedantic'
./configure --enable-devel --enable-werror --disable-optimization
#--enable-sharedptr-debug #--enable-refcnt-debug
librdkafka-0.9.3/examples/ 0000775 0000000 0000000 00000000000 13041406524 0015432 5 ustar 00root root 0000000 0000000 librdkafka-0.9.3/examples/.gitignore 0000664 0000000 0000000 00000000213 13041406524 0017416 0 ustar 00root root 0000000 0000000 rdkafka_example
rdkafka_performance
rdkafka_example_cpp
rdkafka_consumer_example
rdkafka_consumer_example_cpp
kafkatest_verifiable_client
librdkafka-0.9.3/examples/Makefile 0000664 0000000 0000000 00000005453 13041406524 0017101 0 ustar 00root root 0000000 0000000 EXAMPLES ?= rdkafka_example rdkafka_performance rdkafka_example_cpp \
rdkafka_consumer_example rdkafka_consumer_example_cpp \
kafkatest_verifiable_client
all: $(EXAMPLES)
include ../mklove/Makefile.base
CFLAGS += -I../src
CXXFLAGS += -I../src-cpp
# librdkafka must be compiled with -gstrict-dwarf, but rdkafka_example must not,
# due to some clang bug on OSX 10.9
CPPFLAGS := $(subst strict-dwarf,,$(CPPFLAGS))
rdkafka_example: ../src/librdkafka.a rdkafka_example.c
$(CC) $(CPPFLAGS) $(CFLAGS) rdkafka_example.c -o $@ $(LDFLAGS) \
../src/librdkafka.a $(LIBS)
@echo "# $@ is ready"
@echo "#"
@echo "# Run producer (write messages on stdin)"
@echo "./$@ -P -t -p "
@echo ""
@echo "# or consumer"
@echo "./$@ -C -t -p "
@echo ""
@echo "#"
@echo "# More usage options:"
@echo "./$@ -h"
rdkafka_consumer_example: ../src/librdkafka.a rdkafka_consumer_example.c
$(CC) $(CPPFLAGS) $(CFLAGS) rdkafka_consumer_example.c -o $@ $(LDFLAGS) \
../src/librdkafka.a $(LIBS)
@echo "# $@ is ready"
@echo "#"
@echo "./$@ .."
@echo ""
@echo "#"
@echo "# More usage options:"
@echo "./$@ -h"
rdkafka_performance: ../src/librdkafka.a rdkafka_performance.c
$(CC) $(CPPFLAGS) $(CFLAGS) rdkafka_performance.c -o $@ $(LDFLAGS) \
../src/librdkafka.a $(LIBS)
@echo "# $@ is ready"
@echo "#"
@echo "# Run producer"
@echo "./$@ -P -t -p -s "
@echo ""
@echo "# or consumer"
@echo "./$@ -C -t -p "
@echo ""
@echo "#"
@echo "# More usage options:"
@echo "./$@ -h"
rdkafka_example_cpp: ../src-cpp/librdkafka++.a ../src/librdkafka.a rdkafka_example.cpp
$(CXX) $(CPPFLAGS) $(CXXFLAGS) rdkafka_example.cpp -o $@ $(LDFLAGS) \
../src-cpp/librdkafka++.a ../src/librdkafka.a $(LIBS) -lstdc++
kafkatest_verifiable_client: ../src-cpp/librdkafka++.a ../src/librdkafka.a kafkatest_verifiable_client.cpp
$(CXX) $(CPPFLAGS) $(CXXFLAGS) kafkatest_verifiable_client.cpp -o $@ $(LDFLAGS) \
../src-cpp/librdkafka++.a ../src/librdkafka.a $(LIBS) -lstdc++
rdkafka_consumer_example_cpp: ../src-cpp/librdkafka++.a ../src/librdkafka.a rdkafka_consumer_example.cpp
$(CXX) $(CPPFLAGS) $(CXXFLAGS) rdkafka_consumer_example.cpp -o $@ $(LDFLAGS) \
../src-cpp/librdkafka++.a ../src/librdkafka.a $(LIBS) -lstdc++
rdkafka_zookeeper_example: ../src/librdkafka.a rdkafka_zookeeper_example.c
$(CC) $(CPPFLAGS) $(CFLAGS) -I/usr/include/zookeeper rdkafka_zookeeper_example.c -o $@ $(LDFLAGS) \
../src/librdkafka.a $(LIBS) -lzookeeper_mt -ljansson
@echo "# $@ is ready"
@echo "#"
@echo "# Run producer (write messages on stdin)"
@echo "./$@ -P -t -p "
@echo ""
@echo "# or consumer"
@echo "./$@ -C -t -p "
@echo ""
@echo "#"
@echo "# More usage options:"
@echo "./$@ -h"
clean:
rm -f $(EXAMPLES)
librdkafka-0.9.3/examples/globals.json 0000664 0000000 0000000 00000000637 13041406524 0017756 0 ustar 00root root 0000000 0000000 {"VerifiableConsumer":
{
"class": "kafkatest.services.verifiable_client.VerifiableClientApp",
"exec_cmd": "/vagrant/tests/c/kafkatest_verifiable_client --consumer --debug cgrp,topic,protocol,broker"
},
"VerifiableProducer":
{
"class": "kafkatest.services.verifiable_client.VerifiableClientApp",
"exec_cmd": "/vagrant/tests/c/kafkatest_verifiable_client --producer --debug topic,broker"
}
}
librdkafka-0.9.3/examples/kafkatest_verifiable_client.cpp 0000664 0000000 0000000 00000062651 13041406524 0023653 0 ustar 00root root 0000000 0000000 /*
* Copyright (c) 2015, Confluent Inc
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* 1. Redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright notice,
* this list of conditions and the following disclaimer in the documentation
* and/or other materials provided with the distribution.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*/
/**
* librdkafka version of the Java VerifiableProducer and VerifiableConsumer
* for use with the official Kafka client tests.
*/
#include
#include
#include
#include
#include
#include
#include
#include
#include
#include
#include
#include
#include
#include
#include
#ifdef _MSC_VER
#include "../win32/wingetopt.h"
#elif _AIX
#include
#else
#include
#endif
/*
* Typically include path in a real application would be
* #include
*/
#include "rdkafkacpp.h"
static bool run = true;
static bool exit_eof = false;
static int verbosity = 1;
static std::string value_prefix;
class Assignment {
public:
static std::string name (const std::string &t, int partition) {
std::stringstream stm;
stm << t << "." << partition;
return stm.str();
}
Assignment(): topic(""), partition(-1), consumedMessages(0),
minOffset(-1), maxOffset(0) {
printf("Created assignment\n");
}
Assignment(const Assignment &a) {
topic = a.topic;
partition = a.partition;
consumedMessages = a.consumedMessages;
minOffset = a.minOffset;
maxOffset = a.maxOffset;
}
Assignment &operator=(const Assignment &a) {
this->topic = a.topic;
this->partition = a.partition;
this->consumedMessages = a.consumedMessages;
this->minOffset = a.minOffset;
this->maxOffset = a.maxOffset;
return *this;
}
int operator==(const Assignment &a) const {
return !(this->topic == a.topic &&
this->partition == a.partition);
}
int operator<(const Assignment &a) const {
if (this->topic < a.topic) return 1;
if (this->topic >= a.topic) return 0;
return (this->partition < a.partition);
}
void setup (std::string t, int32_t p) {
assert(!t.empty());
assert(topic.empty() || topic == t);
assert(partition == -1 || partition == p);
topic = t;
partition = p;
}
std::string topic;
int partition;
int consumedMessages;
int64_t minOffset;
int64_t maxOffset;
};
static struct {
int maxMessages;
struct {
int numAcked;
int numSent;
int numErr;
} producer;
struct {
int consumedMessages;
int consumedMessagesLastReported;
int consumedMessagesAtLastCommit;
bool useAutoCommit;
bool useAsyncCommit;
std::map assignments;
} consumer;
} state = {
/* .maxMessages = */ -1
};
static RdKafka::KafkaConsumer *consumer;
static std::string now () {
struct timeval tv;
gettimeofday(&tv, NULL);
time_t t = tv.tv_sec;
struct tm *tm = localtime(&t);
char buf[64];
strftime(buf, sizeof(buf), "%H:%M:%S", tm);
snprintf(buf+strlen(buf), sizeof(buf)-strlen(buf), ".%03d",
(int)(tv.tv_usec / 1000));
return buf;
}
static time_t watchdog_last_kick;
static const int watchdog_timeout = 20; /* Must be > socket.timeout.ms */
static void sigwatchdog (int sig) {
time_t t = time(NULL);
if (watchdog_last_kick + watchdog_timeout <= t) {
std::cerr << now() << ": WATCHDOG TIMEOUT (" <<
(int)(t - watchdog_last_kick) << "s): TERMINATING" << std::endl;
int *i = NULL;
*i = 100;
abort();
}
}
static void watchdog_kick () {
watchdog_last_kick = time(NULL);
/* Safe guard against hangs-on-exit */
alarm(watchdog_timeout);
}
static void errorString (const std::string &name,
const std::string &errmsg,
const std::string &topic,
const std::string *key,
const std::string &value) {
std::cout << "{ "
<< "\"name\": \"" << name << "\", "
<< "\"_time\": \"" << now() << "\", "
<< "\"message\": \"" << errmsg << "\", "
<< "\"topic\": \"" << topic << "\", "
<< "\"key\": \"" << (key ? *key : "NULL") << "\", "
<< "\"value\": \"" << value << "\" "
<< "}" << std::endl;
}
static void successString (const std::string &name,
const std::string &topic,
int partition,
int64_t offset,
const std::string *key,
const std::string &value) {
std::cout << "{ "
<< "\"name\": \"" << name << "\", "
<< "\"_time\": \"" << now() << "\", "
<< "\"topic\": \"" << topic << "\", "
<< "\"partition\": " << partition << ", "
<< "\"offset\": " << offset << ", "
<< "\"key\": \"" << (key ? *key : "NULL") << "\", "
<< "\"value\": \"" << value << "\" "
<< "}" << std::endl;
}
#if FIXME
static void offsetStatus (bool success,
const std::string &topic,
int partition,
int64_t offset,
const std::string &errstr) {
std::cout << "{ "
"\"name\": \"offsets_committed\", " <<
"\"success\": " << success << ", " <<
"\"offsets\": [ " <<
" { " <<
" \"topic\": \"" << topic << "\", " <<
" \"partition\": " << partition << ", " <<
" \"offset\": " << (int)offset << ", " <<
" \"error\": \"" << errstr << "\" " <<
" } " <<
"] }" << std::endl;
}
#endif
static void sigterm (int sig) {
std::cerr << now() << ": Terminating because of signal " << sig << std::endl;
if (!run) {
std::cerr << now() << ": Forced termination" << std::endl;
exit(1);
}
run = false;
}
class ExampleDeliveryReportCb : public RdKafka::DeliveryReportCb {
public:
void dr_cb (RdKafka::Message &message) {
if (message.err()) {
state.producer.numErr++;
errorString("producer_send_error", message.errstr(),
message.topic_name(),
message.key(),
std::string(static_cast(message.payload()),
message.len()));
} else {
successString("producer_send_success",
message.topic_name(),
(int)message.partition(),
message.offset(),
message.key(),
std::string(static_cast(message.payload()),
message.len()));
state.producer.numAcked++;
}
}
};
class ExampleEventCb : public RdKafka::EventCb {
public:
void event_cb (RdKafka::Event &event) {
switch (event.type())
{
case RdKafka::Event::EVENT_ERROR:
std::cerr << now() << ": ERROR (" << RdKafka::err2str(event.err()) << "): " <<
event.str() << std::endl;
break;
case RdKafka::Event::EVENT_STATS:
std::cerr << now() << ": \"STATS\": " << event.str() << std::endl;
break;
case RdKafka::Event::EVENT_LOG:
std::cerr << now() << ": LOG-" << event.severity() << "-"
<< event.fac() << ": " << event.str() << std::endl;
break;
default:
std::cerr << now() << ": EVENT " << event.type() <<
" (" << RdKafka::err2str(event.err()) << "): " <<
event.str() << std::endl;
break;
}
}
};
/* Use of this partitioner is pretty pointless since no key is provided
* in the produce() call. */
class MyHashPartitionerCb : public RdKafka::PartitionerCb {
public:
int32_t partitioner_cb (const RdKafka::Topic *topic, const std::string *key,
int32_t partition_cnt, void *msg_opaque) {
return djb_hash(key->c_str(), key->size()) % partition_cnt;
}
private:
static inline unsigned int djb_hash (const char *str, size_t len) {
unsigned int hash = 5381;
for (size_t i = 0 ; i < len ; i++)
hash = ((hash << 5) + hash) + str[i];
return hash;
}
};
/**
* Print number of records consumed, every 100 messages or on timeout.
*/
static void report_records_consumed (int immediate) {
std::map *assignments = &state.consumer.assignments;
if (state.consumer.consumedMessages <=
state.consumer.consumedMessagesLastReported + (immediate ? 0 : 999))
return;
std::cout << "{ "
"\"name\": \"records_consumed\", " <<
"\"_totcount\": " << state.consumer.consumedMessages << ", " <<
"\"count\": " << (state.consumer.consumedMessages -
state.consumer.consumedMessagesLastReported) << ", " <<
"\"partitions\": [ ";
for (std::map::iterator ii = assignments->begin() ;
ii != assignments->end() ; ii++) {
Assignment *a = &(*ii).second;
assert(!a->topic.empty());
std::cout << (ii == assignments->begin() ? "": ", ") << " { " <<
" \"topic\": \"" << a->topic << "\", " <<
" \"partition\": " << a->partition << ", " <<
" \"minOffset\": " << a->minOffset << ", " <<
" \"maxOffset\": " << a->maxOffset << " " <<
" } ";
a->minOffset = -1;
}
std::cout << "] }" << std::endl;
state.consumer.consumedMessagesLastReported = state.consumer.consumedMessages;
}
/**
* Commit every 1000 messages or whenever there is a consume timeout.
* @returns 1 if commit was triggered, else 0.
*/
static int do_commit (RdKafka::KafkaConsumer *consumer,
int immediate) {
if (!state.consumer.useAutoCommit &&
(state.consumer.consumedMessagesAtLastCommit + (immediate ? 0 : 1000)) <
state.consumer.consumedMessages) {
/* Make sure we report consumption before commit,
* otherwise tests may fail because of commit > consumed. */
if (state.consumer.consumedMessagesLastReported <
state.consumer.consumedMessages)
report_records_consumed(1);
std::cerr << now() << ": committing " <<
(state.consumer.consumedMessages -
state.consumer.consumedMessagesAtLastCommit) << " messages (" <<
(state.consumer.useAsyncCommit ? "async":"sync") << ")" << std::endl;
if (state.consumer.useAsyncCommit)
consumer->commitAsync();
else
consumer->commitSync();
std::cerr << now() << ": commit done" << std::endl;
state.consumer.consumedMessagesAtLastCommit =
state.consumer.consumedMessages;
return 1;
} else
return 0;
}
void msg_consume(RdKafka::KafkaConsumer *consumer,
RdKafka::Message* msg, void* opaque) {
switch (msg->err()) {
case RdKafka::ERR__TIMED_OUT:
/* Try reporting consumed messages */
report_records_consumed(1);
/* Commit one every consume() timeout instead of on every message.
* Also commit on every 1000 messages, whichever comes first. */
do_commit(consumer, 1);
break;
case RdKafka::ERR_NO_ERROR:
{
/* Real message */
if (verbosity > 2)
std::cerr << now() << ": Read msg from " << msg->topic_name() <<
" [" << (int)msg->partition() << "] at offset " <<
msg->offset() << std::endl;
if (state.maxMessages >= 0 &&
state.consumer.consumedMessages >= state.maxMessages)
return;
Assignment *a =
&state.consumer.assignments[Assignment::name(msg->topic_name(),
msg->partition())];
a->setup(msg->topic_name(), msg->partition());
a->consumedMessages++;
if (a->minOffset == -1)
a->minOffset = msg->offset();
if (a->maxOffset < msg->offset())
a->maxOffset = msg->offset();
if (msg->key()) {
if (verbosity >= 3)
std::cerr << now() << ": Key: " << *msg->key() << std::endl;
}
if (verbosity >= 3)
fprintf(stderr, "%.*s\n",
static_cast(msg->len()),
static_cast(msg->payload()));
state.consumer.consumedMessages++;
report_records_consumed(0);
do_commit(consumer, 0);
}
break;
case RdKafka::ERR__PARTITION_EOF:
/* Last message */
if (exit_eof) {
std::cerr << now() << ": Termiate: exit on EOF" << std::endl;
run = false;
}
break;
case RdKafka::ERR__UNKNOWN_TOPIC:
case RdKafka::ERR__UNKNOWN_PARTITION:
std::cerr << now() << ": Consume failed: " << msg->errstr() << std::endl;
run = false;
break;
case RdKafka::ERR_GROUP_COORDINATOR_NOT_AVAILABLE:
std::cerr << now() << ": Warning: " << msg->errstr() << std::endl;
break;
default:
/* Errors */
std::cerr << now() << ": Consume failed: " << msg->errstr() << std::endl;
run = false;
}
}
static std::ostringstream deferred_reports;
class ExampleConsumeCb : public RdKafka::ConsumeCb {
public:
void consume_cb (RdKafka::Message &msg, void *opaque) {
msg_consume(consumer_, &msg, opaque);
}
RdKafka::KafkaConsumer *consumer_;
};
class ExampleRebalanceCb : public RdKafka::RebalanceCb {
private:
static std::string part_list_json (const std::vector &partitions) {
std::ostringstream out;
for (unsigned int i = 0 ; i < partitions.size() ; i++)
out << (i==0?"":", ") << "{ " <<
" \"topic\": \"" << partitions[i]->topic() << "\", " <<
" \"partition\": " << partitions[i]->partition() <<
" }";
return out.str();
}
public:
void rebalance_cb (RdKafka::KafkaConsumer *consumer,
RdKafka::ErrorCode err,
std::vector &partitions) {
int deferr = 0;
std::cerr << now() << ": rebalance_cb " << RdKafka::err2str(err) <<
" for " << partitions.size() << " partitions" << std::endl;
/* Send message report prior to rebalancing event to make sure they
* are accounted for on the "right side" of the rebalance. */
report_records_consumed(1);
if (err == RdKafka::ERR__ASSIGN_PARTITIONS)
consumer->assign(partitions);
else {
/* Deferr report to after async auto offset commit? */
deferr = !do_commit(consumer, 1);
consumer->unassign();
}
deferred_reports <<
"{ " <<
"\"name\": \"partitions_" << (err == RdKafka::ERR__ASSIGN_PARTITIONS ?
"assigned" : "revoked") << "\", " <<
"\"partitions\": [ " << part_list_json(partitions) << "] }" << std::endl;
if (!deferr) {
std::cerr << now() << ": Deferring until later: " << deferred_reports.str();
std::cout << deferred_reports.str();
deferred_reports.str("");
}
}
};
class ExampleOffsetCommitCb : public RdKafka::OffsetCommitCb {
public:
void offset_commit_cb (RdKafka::ErrorCode err,
std::vector &offsets) {
std::cerr << now() << ": Propagate offset for " << offsets.size() << " partitions, error: " << RdKafka::err2str(err) << std::endl;
/* No offsets to commit, dont report anything. */
if (err == RdKafka::ERR__NO_OFFSET)
return;
/* Send up-to-date records_consumed report to make sure consumed > committed */
report_records_consumed(1);
std::cout << "{ " <<
"\"name\": \"offsets_committed\", " <<
"\"success\": " << (err ? "false" : "true") << ", " <<
"\"error\": \"" << (err ? RdKafka::err2str(err) : "") << "\", " <<
"\"offsets\": [ ";
assert(offsets.size() > 0);
for (unsigned int i = 0 ; i < offsets.size() ; i++) {
std::cout << (i == 0 ? "" : ", ") << "{ " <<
" \"topic\": \"" << offsets[i]->topic() << "\", " <<
" \"partition\": " << offsets[i]->partition() << ", " <<
" \"offset\": " << (int)offsets[i]->offset() << ", " <<
" \"error\": \"" <<
(offsets[i]->err() ? RdKafka::err2str(offsets[i]->err()) : "") <<
"\" " <<
" }";
}
std::cout << " ] }" << std::endl;
/* Write any deferred reports that needs to be synchronized to
* be written after offsets_committed, such as partitions_revoked. */
std::cout << deferred_reports.str();
deferred_reports.str("");
}
};
static void read_conf_file (const std::string &conf_file) {
std::ifstream inf(conf_file.c_str());
std::string line;
while (std::getline(inf, line)) {
std::cerr << now() << ": conf_file: " << conf_file << ": " << line << std::endl;
}
inf.close();
}
int main (int argc, char **argv) {
std::string brokers = "localhost";
std::string errstr;
std::vector topics;
std::string conf_file;
std::string mode = "P";
int throughput = 0;
int32_t partition = RdKafka::Topic::PARTITION_UA;
bool do_conf_dump = false;
MyHashPartitionerCb hash_partitioner;
/*
* Create configuration objects
*/
RdKafka::Conf *conf = RdKafka::Conf::create(RdKafka::Conf::CONF_GLOBAL);
RdKafka::Conf *tconf = RdKafka::Conf::create(RdKafka::Conf::CONF_TOPIC);
{
char hostname[128];
gethostname(hostname, sizeof(hostname)-1);
conf->set("client.id", std::string("rdkafka@") + hostname, errstr);
}
conf->set("log.thread.name", "true", errstr);
/* auto commit is explicitly enabled with --enable-autocommit */
conf->set("enable.auto.commit", "false", errstr);
/* keep protocol request timeouts under the watchdog timeout
* to make sure things like commitSync() dont fall victim to the watchdog. */
conf->set("socket.timeout.ms", "10000", errstr);
conf->set("fetch.wait.max.ms", "500", errstr);
conf->set("fetch.min.bytes", "4096", errstr);
for (int i = 1 ; i < argc ; i++) {
const char *name = argv[i];
const char *val = i+1 < argc ? argv[i+1] : NULL;
if (val && !strncmp(val, "-", 1))
val = NULL;
std::cout << now() << ": argument: " << name << " " <<
(val?val:"") << std::endl;
if (val) {
if (!strcmp(name, "--topic"))
topics.push_back(val);
else if (!strcmp(name, "--broker-list"))
brokers = val;
else if (!strcmp(name, "--max-messages"))
state.maxMessages = atoi(val);
else if (!strcmp(name, "--throughput"))
throughput = atoi(val);
else if (!strcmp(name, "--producer.config") ||
!strcmp(name, "--consumer.config"))
read_conf_file(val);
else if (!strcmp(name, "--group-id"))
conf->set("group.id", val, errstr);
else if (!strcmp(name, "--session-timeout"))
conf->set("session.timeout.ms", val, errstr);
else if (!strcmp(name, "--reset-policy")) {
if (tconf->set("auto.offset.reset", val, errstr)) {
std::cerr << now() << ": " << errstr << std::endl;
exit(1);
}
} else if (!strcmp(name, "--assignment-strategy")) {
/* The system tests pass the Java class name(s) rather than
* the configuration value. Fix it.
* "org.apache.kafka.clients.consumer.RangeAssignor,.." -> "range,.."
*/
std::string s = val;
size_t pos;
while ((pos = s.find("org.apache.kafka.clients.consumer.")) !=
std::string::npos)
s.erase(pos, strlen("org.apache.kafka.clients.consumer."));
while ((pos = s.find("Assignor")) != std::string::npos)
s.erase(pos, strlen("Assignor"));
std::transform(s.begin(), s.end(), s.begin(), tolower);
std::cerr << now() << ": converted " << name << " "
<< val << " to " << s << std::endl;
if (conf->set("partition.assignment.strategy", s.c_str(), errstr)) {
std::cerr << now() << ": " << errstr << std::endl;
exit(1);
}
} else if (!strcmp(name, "--value-prefix")) {
value_prefix = std::string(val) + ".";
} else if (!strcmp(name, "--debug")) {
conf->set("debug", val, errstr);
} else {
std::cerr << now() << ": Unknown option " << name << std::endl;
exit(1);
}
i++;
} else {
if (!strcmp(name, "--consumer"))
mode = "C";
else if (!strcmp(name, "--producer"))
mode = "P";
else if (!strcmp(name, "--enable-autocommit")) {
state.consumer.useAutoCommit = true;
conf->set("enable.auto.commit", "true", errstr);
} else if (!strcmp(name, "-v"))
verbosity++;
else if (!strcmp(name, "-q"))
verbosity--;
else {
std::cerr << now() << ": Unknown option or missing argument to " << name << std::endl;
exit(1);
}
}
}
if (topics.empty() || brokers.empty()) {
std::cerr << now() << ": Missing --topic and --broker-list" << std::endl;
exit(1);
}
/*
* Set configuration properties
*/
conf->set("metadata.broker.list", brokers, errstr);
ExampleEventCb ex_event_cb;
conf->set("event_cb", &ex_event_cb, errstr);
if (do_conf_dump) {
int pass;
for (pass = 0 ; pass < 2 ; pass++) {
std::list *dump;
if (pass == 0) {
dump = conf->dump();
std::cerr << now() << ": # Global config" << std::endl;
} else {
dump = tconf->dump();
std::cerr << now() << ": # Topic config" << std::endl;
}
for (std::list::iterator it = dump->begin();
it != dump->end(); ) {
std::cerr << *it << " = ";
it++;
std::cerr << *it << std::endl;
it++;
}
std::cerr << std::endl;
}
exit(0);
}
signal(SIGINT, sigterm);
signal(SIGTERM, sigterm);
signal(SIGALRM, sigwatchdog);
if (mode == "P") {
/*
* Producer mode
*/
ExampleDeliveryReportCb ex_dr_cb;
/* Set delivery report callback */
conf->set("dr_cb", &ex_dr_cb, errstr);
/*
* Create producer using accumulated global configuration.
*/
RdKafka::Producer *producer = RdKafka::Producer::create(conf, errstr);
if (!producer) {
std::cerr << now() << ": Failed to create producer: " << errstr << std::endl;
exit(1);
}
std::cerr << now() << ": % Created producer " << producer->name() << std::endl;
/*
* Create topic handle.
*/
RdKafka::Topic *topic = RdKafka::Topic::create(producer, topics[0],
tconf, errstr);
if (!topic) {
std::cerr << now() << ": Failed to create topic: " << errstr << std::endl;
exit(1);
}
static const int delay_us = throughput ? 1000000/throughput : 0;
if (state.maxMessages == -1)
state.maxMessages = 1000000; /* Avoid infinite produce */
for (int i = 0 ; run && i < state.maxMessages ; i++) {
/*
* Produce message
*/
std::ostringstream msg;
msg << value_prefix << i;
RdKafka::ErrorCode resp =
producer->produce(topic, partition,
RdKafka::Producer::RK_MSG_COPY /* Copy payload */,
const_cast(msg.str().c_str()),
msg.str().size(), NULL, NULL);
if (resp != RdKafka::ERR_NO_ERROR) {
errorString("producer_send_error",
RdKafka::err2str(resp), topic->name(), NULL, msg.str());
state.producer.numErr++;
} else {
std::cerr << now() << ": % Produced message (" <<
msg.str().size() << " bytes)" << std::endl;
state.producer.numSent++;
}
producer->poll(delay_us / 1000);
watchdog_kick();
}
run = true;
while (run && producer->outq_len() > 0) {
std::cerr << now() << ": Waiting for " << producer->outq_len() << std::endl;
producer->poll(50);
watchdog_kick();
}
std::cerr << now() << ": " << state.producer.numAcked << "/" <<
state.producer.numSent << "/" << state.maxMessages <<
" msgs acked/sent/max, " << state.producer.numErr <<
" errored" << std::endl;
delete topic;
delete producer;
} else if (mode == "C") {
/*
* Consumer mode
*/
tconf->set("auto.offset.reset", "smallest", errstr);
/* Set default topic config */
conf->set("default_topic_conf", tconf, errstr);
ExampleRebalanceCb ex_rebalance_cb;
conf->set("rebalance_cb", &ex_rebalance_cb, errstr);
ExampleOffsetCommitCb ex_offset_commit_cb;
conf->set("offset_commit_cb", &ex_offset_commit_cb, errstr);
/*
* Create consumer using accumulated global configuration.
*/
consumer = RdKafka::KafkaConsumer::create(conf, errstr);
if (!consumer) {
std::cerr << now() << ": Failed to create consumer: " <<
errstr << std::endl;
exit(1);
}
std::cerr << now() << ": % Created consumer " << consumer->name() <<
std::endl;
/*
* Subscribe to topic(s)
*/
RdKafka::ErrorCode resp = consumer->subscribe(topics);
if (resp != RdKafka::ERR_NO_ERROR) {
std::cerr << now() << ": Failed to subscribe to " << topics.size() << " topics: "
<< RdKafka::err2str(resp) << std::endl;
exit(1);
}
watchdog_kick();
/*
* Consume messages
*/
while (run) {
RdKafka::Message *msg = consumer->consume(500);
msg_consume(consumer, msg, NULL);
delete msg;
watchdog_kick();
}
std::cerr << now() << ": Final commit on termination" << std::endl;
/* Final commit */
do_commit(consumer, 1);
/*
* Stop consumer
*/
consumer->close();
delete consumer;
}
std::cout << "{ \"name\": \"shutdown_complete\" }" << std::endl;
/*
* Wait for RdKafka to decommission.
* This is not strictly needed (when check outq_len() above), but
* allows RdKafka to clean up all its resources before the application
* exits so that memory profilers such as valgrind wont complain about
* memory leaks.
*/
RdKafka::wait_destroyed(5000);
std::cerr << now() << ": EXITING WITH RETURN VALUE 0" << std::endl;
return 0;
}
librdkafka-0.9.3/examples/rdkafka_consumer_example.c 0000664 0000000 0000000 00000045175 13041406524 0022643 0 ustar 00root root 0000000 0000000 /*
* librdkafka - Apache Kafka C library
*
* Copyright (c) 2015, Magnus Edenhill
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* 1. Redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright notice,
* this list of conditions and the following disclaimer in the documentation
* and/or other materials provided with the distribution.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*/
/**
* Apache Kafka high level consumer example program
* using the Kafka driver from librdkafka
* (https://github.com/edenhill/librdkafka)
*/
#include
#include
#include
#include
#include
#include
#include
#include
#include
/* Typical include path would be , but this program
* is builtin from within the librdkafka source tree and thus differs. */
#include "rdkafka.h" /* for Kafka driver */
static int run = 1;
static rd_kafka_t *rk;
static int exit_eof = 0;
static int wait_eof = 0; /* number of partitions awaiting EOF */
static int quiet = 0;
static enum {
OUTPUT_HEXDUMP,
OUTPUT_RAW,
} output = OUTPUT_HEXDUMP;
static void stop (int sig) {
if (!run)
exit(1);
run = 0;
fclose(stdin); /* abort fgets() */
}
static void hexdump (FILE *fp, const char *name, const void *ptr, size_t len) {
const char *p = (const char *)ptr;
unsigned int of = 0;
if (name)
fprintf(fp, "%s hexdump (%zd bytes):\n", name, len);
for (of = 0 ; of < len ; of += 16) {
char hexen[16*3+1];
char charen[16+1];
int hof = 0;
int cof = 0;
int i;
for (i = of ; i < (int)of + 16 && i < (int)len ; i++) {
hof += sprintf(hexen+hof, "%02x ", p[i] & 0xff);
cof += sprintf(charen+cof, "%c",
isprint((int)p[i]) ? p[i] : '.');
}
fprintf(fp, "%08x: %-48s %-16s\n",
of, hexen, charen);
}
}
/**
* Kafka logger callback (optional)
*/
static void logger (const rd_kafka_t *rk, int level,
const char *fac, const char *buf) {
struct timeval tv;
gettimeofday(&tv, NULL);
fprintf(stdout, "%u.%03u RDKAFKA-%i-%s: %s: %s\n",
(int)tv.tv_sec, (int)(tv.tv_usec / 1000),
level, fac, rd_kafka_name(rk), buf);
}
/**
* Handle and print a consumed message.
* Internally crafted messages are also used to propagate state from
* librdkafka to the application. The application needs to check
* the `rkmessage->err` field for this purpose.
*/
static void msg_consume (rd_kafka_message_t *rkmessage,
void *opaque) {
if (rkmessage->err) {
if (rkmessage->err == RD_KAFKA_RESP_ERR__PARTITION_EOF) {
fprintf(stderr,
"%% Consumer reached end of %s [%"PRId32"] "
"message queue at offset %"PRId64"\n",
rd_kafka_topic_name(rkmessage->rkt),
rkmessage->partition, rkmessage->offset);
if (exit_eof && --wait_eof == 0) {
fprintf(stderr,
"%% All partition(s) reached EOF: "
"exiting\n");
run = 0;
}
return;
}
if (rkmessage->rkt)
fprintf(stderr, "%% Consume error for "
"topic \"%s\" [%"PRId32"] "
"offset %"PRId64": %s\n",
rd_kafka_topic_name(rkmessage->rkt),
rkmessage->partition,
rkmessage->offset,
rd_kafka_message_errstr(rkmessage));
else
fprintf(stderr, "%% Consumer error: %s: %s\n",
rd_kafka_err2str(rkmessage->err),
rd_kafka_message_errstr(rkmessage));
if (rkmessage->err == RD_KAFKA_RESP_ERR__UNKNOWN_PARTITION ||
rkmessage->err == RD_KAFKA_RESP_ERR__UNKNOWN_TOPIC)
run = 0;
return;
}
if (!quiet)
fprintf(stdout, "%% Message (topic %s [%"PRId32"], "
"offset %"PRId64", %zd bytes):\n",
rd_kafka_topic_name(rkmessage->rkt),
rkmessage->partition,
rkmessage->offset, rkmessage->len);
if (rkmessage->key_len) {
if (output == OUTPUT_HEXDUMP)
hexdump(stdout, "Message Key",
rkmessage->key, rkmessage->key_len);
else
printf("Key: %.*s\n",
(int)rkmessage->key_len, (char *)rkmessage->key);
}
if (output == OUTPUT_HEXDUMP)
hexdump(stdout, "Message Payload",
rkmessage->payload, rkmessage->len);
else
printf("%.*s\n",
(int)rkmessage->len, (char *)rkmessage->payload);
}
static void print_partition_list (FILE *fp,
const rd_kafka_topic_partition_list_t
*partitions) {
int i;
for (i = 0 ; i < partitions->cnt ; i++) {
fprintf(stderr, "%s %s [%"PRId32"] offset %"PRId64,
i > 0 ? ",":"",
partitions->elems[i].topic,
partitions->elems[i].partition,
partitions->elems[i].offset);
}
fprintf(stderr, "\n");
}
static void rebalance_cb (rd_kafka_t *rk,
rd_kafka_resp_err_t err,
rd_kafka_topic_partition_list_t *partitions,
void *opaque) {
fprintf(stderr, "%% Consumer group rebalanced: ");
switch (err)
{
case RD_KAFKA_RESP_ERR__ASSIGN_PARTITIONS:
fprintf(stderr, "assigned:\n");
print_partition_list(stderr, partitions);
rd_kafka_assign(rk, partitions);
wait_eof += partitions->cnt;
break;
case RD_KAFKA_RESP_ERR__REVOKE_PARTITIONS:
fprintf(stderr, "revoked:\n");
print_partition_list(stderr, partitions);
rd_kafka_assign(rk, NULL);
wait_eof = 0;
break;
default:
fprintf(stderr, "failed: %s\n",
rd_kafka_err2str(err));
rd_kafka_assign(rk, NULL);
break;
}
}
static int describe_groups (rd_kafka_t *rk, const char *group) {
rd_kafka_resp_err_t err;
const struct rd_kafka_group_list *grplist;
int i;
err = rd_kafka_list_groups(rk, group, &grplist, 10000);
if (err) {
fprintf(stderr, "%% Failed to acquire group list: %s\n",
rd_kafka_err2str(err));
return -1;
}
for (i = 0 ; i < grplist->group_cnt ; i++) {
const struct rd_kafka_group_info *gi = &grplist->groups[i];
int j;
printf("Group \"%s\" in state %s on broker %d (%s:%d)\n",
gi->group, gi->state,
gi->broker.id, gi->broker.host, gi->broker.port);
if (gi->err)
printf(" Error: %s\n", rd_kafka_err2str(gi->err));
printf(" Protocol type \"%s\", protocol \"%s\", "
"with %d member(s):\n",
gi->protocol_type, gi->protocol, gi->member_cnt);
for (j = 0 ; j < gi->member_cnt ; j++) {
const struct rd_kafka_group_member_info *mi;
mi = &gi->members[j];
printf(" \"%s\", client id \"%s\" on host %s\n",
mi->member_id, mi->client_id, mi->client_host);
printf(" metadata: %d bytes\n",
mi->member_metadata_size);
printf(" assignment: %d bytes\n",
mi->member_assignment_size);
}
printf("\n");
}
if (group && !grplist->group_cnt)
fprintf(stderr, "%% No matching group (%s)\n", group);
rd_kafka_group_list_destroy(grplist);
return 0;
}
static void sig_usr1 (int sig) {
rd_kafka_dump(stdout, rk);
}
int main (int argc, char **argv) {
char mode = 'C';
char *brokers = "localhost:9092";
int opt;
rd_kafka_conf_t *conf;
rd_kafka_topic_conf_t *topic_conf;
char errstr[512];
const char *debug = NULL;
int do_conf_dump = 0;
char tmp[16];
rd_kafka_resp_err_t err;
char *group = NULL;
rd_kafka_topic_partition_list_t *topics;
int is_subscription;
int i;
quiet = !isatty(STDIN_FILENO);
/* Kafka configuration */
conf = rd_kafka_conf_new();
/* Set logger */
rd_kafka_conf_set_log_cb(conf, logger);
/* Quick termination */
snprintf(tmp, sizeof(tmp), "%i", SIGIO);
rd_kafka_conf_set(conf, "internal.termination.signal", tmp, NULL, 0);
/* Topic configuration */
topic_conf = rd_kafka_topic_conf_new();
while ((opt = getopt(argc, argv, "g:b:qd:eX:As:DO")) != -1) {
switch (opt) {
case 'b':
brokers = optarg;
break;
case 'g':
group = optarg;
break;
case 'e':
exit_eof = 1;
break;
case 'd':
debug = optarg;
break;
case 'q':
quiet = 1;
break;
case 'A':
output = OUTPUT_RAW;
break;
case 'X':
{
char *name, *val;
rd_kafka_conf_res_t res;
if (!strcmp(optarg, "list") ||
!strcmp(optarg, "help")) {
rd_kafka_conf_properties_show(stdout);
exit(0);
}
if (!strcmp(optarg, "dump")) {
do_conf_dump = 1;
continue;
}
name = optarg;
if (!(val = strchr(name, '='))) {
fprintf(stderr, "%% Expected "
"-X property=value, not %s\n", name);
exit(1);
}
*val = '\0';
val++;
res = RD_KAFKA_CONF_UNKNOWN;
/* Try "topic." prefixed properties on topic
* conf first, and then fall through to global if
* it didnt match a topic configuration property. */
if (!strncmp(name, "topic.", strlen("topic.")))
res = rd_kafka_topic_conf_set(topic_conf,
name+
strlen("topic."),
val,
errstr,
sizeof(errstr));
if (res == RD_KAFKA_CONF_UNKNOWN)
res = rd_kafka_conf_set(conf, name, val,
errstr, sizeof(errstr));
if (res != RD_KAFKA_CONF_OK) {
fprintf(stderr, "%% %s\n", errstr);
exit(1);
}
}
break;
case 'D':
case 'O':
mode = opt;
break;
default:
goto usage;
}
}
if (do_conf_dump) {
const char **arr;
size_t cnt;
int pass;
for (pass = 0 ; pass < 2 ; pass++) {
if (pass == 0) {
arr = rd_kafka_conf_dump(conf, &cnt);
printf("# Global config\n");
} else {
printf("# Topic config\n");
arr = rd_kafka_topic_conf_dump(topic_conf,
&cnt);
}
for (i = 0 ; i < (int)cnt ; i += 2)
printf("%s = %s\n",
arr[i], arr[i+1]);
printf("\n");
rd_kafka_conf_dump_free(arr, cnt);
}
exit(0);
}
if (strchr("OC", mode) && optind == argc) {
usage:
fprintf(stderr,
"Usage: %s [options] ..\n"
"\n"
"librdkafka version %s (0x%08x)\n"
"\n"
" Options:\n"
" -g Consumer group (%s)\n"
" -b Broker address (%s)\n"
" -e Exit consumer when last message\n"
" in partition has been received.\n"
" -D Describe group.\n"
" -O Get commmitted offset(s)\n"
" -d [facs..] Enable debugging contexts:\n"
" %s\n"
" -q Be quiet\n"
" -A Raw payload output (consumer)\n"
" -X Set arbitrary librdkafka "
"configuration property\n"
" Properties prefixed with \"topic.\" "
"will be set on topic object.\n"
" Use '-X list' to see the full list\n"
" of supported properties.\n"
"\n"
"For balanced consumer groups use the 'topic1 topic2..'"
" format\n"
"and for static assignment use "
"'topic1:part1 topic1:part2 topic2:part1..'\n"
"\n",
argv[0],
rd_kafka_version_str(), rd_kafka_version(),
group, brokers,
RD_KAFKA_DEBUG_CONTEXTS);
exit(1);
}
signal(SIGINT, stop);
signal(SIGUSR1, sig_usr1);
if (debug &&
rd_kafka_conf_set(conf, "debug", debug, errstr, sizeof(errstr)) !=
RD_KAFKA_CONF_OK) {
fprintf(stderr, "%% Debug configuration failed: %s: %s\n",
errstr, debug);
exit(1);
}
/*
* Client/Consumer group
*/
if (strchr("CO", mode)) {
/* Consumer groups require a group id */
if (!group)
group = "rdkafka_consumer_example";
if (rd_kafka_conf_set(conf, "group.id", group,
errstr, sizeof(errstr)) !=
RD_KAFKA_CONF_OK) {
fprintf(stderr, "%% %s\n", errstr);
exit(1);
}
/* Consumer groups always use broker based offset storage */
if (rd_kafka_topic_conf_set(topic_conf, "offset.store.method",
"broker",
errstr, sizeof(errstr)) !=
RD_KAFKA_CONF_OK) {
fprintf(stderr, "%% %s\n", errstr);
exit(1);
}
/* Set default topic config for pattern-matched topics. */
rd_kafka_conf_set_default_topic_conf(conf, topic_conf);
/* Callback called on partition assignment changes */
rd_kafka_conf_set_rebalance_cb(conf, rebalance_cb);
}
/* Create Kafka handle */
if (!(rk = rd_kafka_new(RD_KAFKA_CONSUMER, conf,
errstr, sizeof(errstr)))) {
fprintf(stderr,
"%% Failed to create new consumer: %s\n",
errstr);
exit(1);
}
rd_kafka_set_log_level(rk, LOG_DEBUG);
/* Add brokers */
if (rd_kafka_brokers_add(rk, brokers) == 0) {
fprintf(stderr, "%% No valid brokers specified\n");
exit(1);
}
if (mode == 'D') {
int r;
/* Describe groups */
r = describe_groups(rk, group);
rd_kafka_destroy(rk);
exit(r == -1 ? 1 : 0);
}
/* Redirect rd_kafka_poll() to consumer_poll() */
rd_kafka_poll_set_consumer(rk);
topics = rd_kafka_topic_partition_list_new(argc - optind);
is_subscription = 1;
for (i = optind ; i < argc ; i++) {
/* Parse "topic[:part] */
char *topic = argv[i];
char *t;
int32_t partition = -1;
if ((t = strstr(topic, ":"))) {
*t = '\0';
partition = atoi(t+1);
is_subscription = 0; /* is assignment */
wait_eof++;
}
rd_kafka_topic_partition_list_add(topics, topic, partition);
}
if (mode == 'O') {
/* Offset query */
err = rd_kafka_committed(rk, topics, 5000);
if (err) {
fprintf(stderr, "%% Failed to fetch offsets: %s\n",
rd_kafka_err2str(err));
exit(1);
}
for (i = 0 ; i < topics->cnt ; i++) {
rd_kafka_topic_partition_t *p = &topics->elems[i];
printf("Topic \"%s\" partition %"PRId32,
p->topic, p->partition);
if (p->err)
printf(" error %s",
rd_kafka_err2str(p->err));
else {
printf(" offset %"PRId64"",
p->offset);
if (p->metadata_size)
printf(" (%d bytes of metadata)",
(int)p->metadata_size);
}
printf("\n");
}
goto done;
}
if (is_subscription) {
fprintf(stderr, "%% Subscribing to %d topics\n", topics->cnt);
if ((err = rd_kafka_subscribe(rk, topics))) {
fprintf(stderr,
"%% Failed to start consuming topics: %s\n",
rd_kafka_err2str(err));
exit(1);
}
} else {
fprintf(stderr, "%% Assigning %d partitions\n", topics->cnt);
if ((err = rd_kafka_assign(rk, topics))) {
fprintf(stderr,
"%% Failed to assign partitions: %s\n",
rd_kafka_err2str(err));
}
}
while (run) {
rd_kafka_message_t *rkmessage;
rkmessage = rd_kafka_consumer_poll(rk, 1000);
if (rkmessage) {
msg_consume(rkmessage, NULL);
rd_kafka_message_destroy(rkmessage);
}
}
done:
err = rd_kafka_consumer_close(rk);
if (err)
fprintf(stderr, "%% Failed to close consumer: %s\n",
rd_kafka_err2str(err));
else
fprintf(stderr, "%% Consumer closed\n");
rd_kafka_topic_partition_list_destroy(topics);
/* Destroy handle */
rd_kafka_destroy(rk);
/* Let background threads clean up and terminate cleanly. */
run = 5;
while (run-- > 0 && rd_kafka_wait_destroyed(1000) == -1)
printf("Waiting for librdkafka to decommission\n");
if (run <= 0)
rd_kafka_dump(stdout, rk);
return 0;
}
librdkafka-0.9.3/examples/rdkafka_consumer_example.cpp 0000664 0000000 0000000 00000031542 13041406524 0023174 0 ustar 00root root 0000000 0000000 /*
* librdkafka - Apache Kafka C library
*
* Copyright (c) 2014, Magnus Edenhill
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* 1. Redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright notice,
* this list of conditions and the following disclaimer in the documentation
* and/or other materials provided with the distribution.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*/
/**
* Apache Kafka consumer & producer example programs
* using the Kafka driver from librdkafka
* (https://github.com/edenhill/librdkafka)
*/
#include
#include
#include
#include
#include
#include
#ifndef _MSC_VER
#include
#endif
#ifdef _MSC_VER
#include "../win32/wingetopt.h"
#include
#elif _AIX
#include
#else
#include
#include
#endif
/*
* Typically include path in a real application would be
* #include
*/
#include "rdkafkacpp.h"
static bool run = true;
static bool exit_eof = false;
static int eof_cnt = 0;
static int partition_cnt = 0;
static int verbosity = 1;
static long msg_cnt = 0;
static int64_t msg_bytes = 0;
static void sigterm (int sig) {
run = false;
}
/**
* @brief format a string timestamp from the current time
*/
static void print_time () {
#ifndef _MSC_VER
struct timeval tv;
char buf[64];
gettimeofday(&tv, NULL);
strftime(buf, sizeof(buf) - 1, "%Y-%m-%d %H:%M:%S", localtime(&tv.tv_sec));
fprintf(stderr, "%s.%03d: ", buf, (int)(tv.tv_usec / 1000));
#else
std::wcerr << CTime::GetCurrentTime().Format(_T("%Y-%m-%d %H:%M:%S")).GetString()
<< ": ";
#endif
}
class ExampleEventCb : public RdKafka::EventCb {
public:
void event_cb (RdKafka::Event &event) {
print_time();
switch (event.type())
{
case RdKafka::Event::EVENT_ERROR:
std::cerr << "ERROR (" << RdKafka::err2str(event.err()) << "): " <<
event.str() << std::endl;
if (event.err() == RdKafka::ERR__ALL_BROKERS_DOWN)
run = false;
break;
case RdKafka::Event::EVENT_STATS:
std::cerr << "\"STATS\": " << event.str() << std::endl;
break;
case RdKafka::Event::EVENT_LOG:
fprintf(stderr, "LOG-%i-%s: %s\n",
event.severity(), event.fac().c_str(), event.str().c_str());
break;
case RdKafka::Event::EVENT_THROTTLE:
std::cerr << "THROTTLED: " << event.throttle_time() << "ms by " <<
event.broker_name() << " id " << (int)event.broker_id() << std::endl;
break;
default:
std::cerr << "EVENT " << event.type() <<
" (" << RdKafka::err2str(event.err()) << "): " <<
event.str() << std::endl;
break;
}
}
};
class ExampleRebalanceCb : public RdKafka::RebalanceCb {
private:
static void part_list_print (const std::vector&partitions){
for (unsigned int i = 0 ; i < partitions.size() ; i++)
std::cerr << partitions[i]->topic() <<
"[" << partitions[i]->partition() << "], ";
std::cerr << "\n";
}
public:
void rebalance_cb (RdKafka::KafkaConsumer *consumer,
RdKafka::ErrorCode err,
std::vector &partitions) {
std::cerr << "RebalanceCb: " << RdKafka::err2str(err) << ": ";
part_list_print(partitions);
if (err == RdKafka::ERR__ASSIGN_PARTITIONS) {
consumer->assign(partitions);
partition_cnt = (int)partitions.size();
} else {
consumer->unassign();
partition_cnt = 0;
}
eof_cnt = 0;
}
};
void msg_consume(RdKafka::Message* message, void* opaque) {
switch (message->err()) {
case RdKafka::ERR__TIMED_OUT:
break;
case RdKafka::ERR_NO_ERROR:
/* Real message */
msg_cnt++;
msg_bytes += message->len();
if (verbosity >= 3)
std::cerr << "Read msg at offset " << message->offset() << std::endl;
RdKafka::MessageTimestamp ts;
ts = message->timestamp();
if (verbosity >= 2 &&
ts.type != RdKafka::MessageTimestamp::MSG_TIMESTAMP_NOT_AVAILABLE) {
std::string tsname = "?";
if (ts.type == RdKafka::MessageTimestamp::MSG_TIMESTAMP_CREATE_TIME)
tsname = "create time";
else if (ts.type == RdKafka::MessageTimestamp::MSG_TIMESTAMP_LOG_APPEND_TIME)
tsname = "log append time";
std::cout << "Timestamp: " << tsname << " " << ts.timestamp << std::endl;
}
if (verbosity >= 2 && message->key()) {
std::cout << "Key: " << *message->key() << std::endl;
}
if (verbosity >= 1) {
printf("%.*s\n",
static_cast(message->len()),
static_cast(message->payload()));
}
break;
case RdKafka::ERR__PARTITION_EOF:
/* Last message */
if (exit_eof && ++eof_cnt == partition_cnt) {
std::cerr << "%% EOF reached for all " << partition_cnt <<
" partition(s)" << std::endl;
run = false;
}
break;
case RdKafka::ERR__UNKNOWN_TOPIC:
case RdKafka::ERR__UNKNOWN_PARTITION:
std::cerr << "Consume failed: " << message->errstr() << std::endl;
run = false;
break;
default:
/* Errors */
std::cerr << "Consume failed: " << message->errstr() << std::endl;
run = false;
}
}
class ExampleConsumeCb : public RdKafka::ConsumeCb {
public:
void consume_cb (RdKafka::Message &msg, void *opaque) {
msg_consume(&msg, opaque);
}
};
int main (int argc, char **argv) {
std::string brokers = "localhost";
std::string errstr;
std::string topic_str;
std::string mode;
std::string debug;
std::vector topics;
bool do_conf_dump = false;
int opt;
int use_ccb = 0;
/*
* Create configuration objects
*/
RdKafka::Conf *conf = RdKafka::Conf::create(RdKafka::Conf::CONF_GLOBAL);
RdKafka::Conf *tconf = RdKafka::Conf::create(RdKafka::Conf::CONF_TOPIC);
ExampleRebalanceCb ex_rebalance_cb;
conf->set("rebalance_cb", &ex_rebalance_cb, errstr);
while ((opt = getopt(argc, argv, "g:b:z:qd:eX:AM:f:qv")) != -1) {
switch (opt) {
case 'g':
if (conf->set("group.id", optarg, errstr) != RdKafka::Conf::CONF_OK) {
std::cerr << errstr << std::endl;
exit(1);
}
break;
case 'b':
brokers = optarg;
break;
case 'z':
if (conf->set("compression.codec", optarg, errstr) !=
RdKafka::Conf::CONF_OK) {
std::cerr << errstr << std::endl;
exit(1);
}
break;
case 'e':
exit_eof = true;
break;
case 'd':
debug = optarg;
break;
case 'M':
if (conf->set("statistics.interval.ms", optarg, errstr) !=
RdKafka::Conf::CONF_OK) {
std::cerr << errstr << std::endl;
exit(1);
}
break;
case 'X':
{
char *name, *val;
if (!strcmp(optarg, "dump")) {
do_conf_dump = true;
continue;
}
name = optarg;
if (!(val = strchr(name, '='))) {
std::cerr << "%% Expected -X property=value, not " <<
name << std::endl;
exit(1);
}
*val = '\0';
val++;
/* Try "topic." prefixed properties on topic
* conf first, and then fall through to global if
* it didnt match a topic configuration property. */
RdKafka::Conf::ConfResult res = RdKafka::Conf::CONF_UNKNOWN;
if (!strncmp(name, "topic.", strlen("topic.")))
res = tconf->set(name+strlen("topic."), val, errstr);
if (res == RdKafka::Conf::CONF_UNKNOWN)
res = conf->set(name, val, errstr);
if (res != RdKafka::Conf::CONF_OK) {
std::cerr << errstr << std::endl;
exit(1);
}
}
break;
case 'f':
if (!strcmp(optarg, "ccb"))
use_ccb = 1;
else {
std::cerr << "Unknown option: " << optarg << std::endl;
exit(1);
}
break;
case 'q':
verbosity--;
break;
case 'v':
verbosity++;
break;
default:
goto usage;
}
}
for (; optind < argc ; optind++)
topics.push_back(std::string(argv[optind]));
if (topics.empty() || optind != argc) {
usage:
fprintf(stderr,
"Usage: %s -g [options] topic1 topic2..\n"
"\n"
"librdkafka version %s (0x%08x)\n"
"\n"
" Options:\n"
" -g Consumer group id\n"
" -b Broker address (localhost:9092)\n"
" -z Enable compression:\n"
" none|gzip|snappy\n"
" -e Exit consumer when last message\n"
" in partition has been received.\n"
" -d [facs..] Enable debugging contexts:\n"
" %s\n"
" -M Enable statistics\n"
" -X Set arbitrary librdkafka "
"configuration property\n"
" Properties prefixed with \"topic.\" "
"will be set on topic object.\n"
" Use '-X list' to see the full list\n"
" of supported properties.\n"
" -f Set option:\n"
" ccb - use consume_callback\n"
" -q Quiet / Decrease verbosity\n"
" -v Increase verbosity\n"
"\n"
"\n",
argv[0],
RdKafka::version_str().c_str(), RdKafka::version(),
RdKafka::get_debug_contexts().c_str());
exit(1);
}
/*
* Set configuration properties
*/
conf->set("metadata.broker.list", brokers, errstr);
if (!debug.empty()) {
if (conf->set("debug", debug, errstr) != RdKafka::Conf::CONF_OK) {
std::cerr << errstr << std::endl;
exit(1);
}
}
ExampleConsumeCb ex_consume_cb;
if(use_ccb) {
conf->set("consume_cb", &ex_consume_cb, errstr);
}
ExampleEventCb ex_event_cb;
conf->set("event_cb", &ex_event_cb, errstr);
conf->set("default_topic_conf", tconf, errstr);
delete tconf;
if (do_conf_dump) {
int pass;
for (pass = 0 ; pass < 2 ; pass++) {
std::list *dump;
if (pass == 0) {
dump = conf->dump();
std::cout << "# Global config" << std::endl;
} else {
dump = tconf->dump();
std::cout << "# Topic config" << std::endl;
}
for (std::list::iterator it = dump->begin();
it != dump->end(); ) {
std::cout << *it << " = ";
it++;
std::cout << *it << std::endl;
it++;
}
std::cout << std::endl;
}
exit(0);
}
signal(SIGINT, sigterm);
signal(SIGTERM, sigterm);
/*
* Consumer mode
*/
/*
* Create consumer using accumulated global configuration.
*/
RdKafka::KafkaConsumer *consumer = RdKafka::KafkaConsumer::create(conf, errstr);
if (!consumer) {
std::cerr << "Failed to create consumer: " << errstr << std::endl;
exit(1);
}
delete conf;
std::cout << "% Created consumer " << consumer->name() << std::endl;
/*
* Subscribe to topics
*/
RdKafka::ErrorCode err = consumer->subscribe(topics);
if (err) {
std::cerr << "Failed to subscribe to " << topics.size() << " topics: "
<< RdKafka::err2str(err) << std::endl;
exit(1);
}
/*
* Consume messages
*/
while (run) {
RdKafka::Message *msg = consumer->consume(1000);
if (!use_ccb) {
msg_consume(msg, NULL);
}
delete msg;
}
#ifndef _MSC_VER
alarm(10);
#endif
/*
* Stop consumer
*/
consumer->close();
delete consumer;
std::cerr << "% Consumed " << msg_cnt << " messages ("
<< msg_bytes << " bytes)" << std::endl;
/*
* Wait for RdKafka to decommission.
* This is not strictly needed (with check outq_len() above), but
* allows RdKafka to clean up all its resources before the application
* exits so that memory profilers such as valgrind wont complain about
* memory leaks.
*/
RdKafka::wait_destroyed(5000);
return 0;
}
librdkafka-0.9.3/examples/rdkafka_example.c 0000664 0000000 0000000 00000055775 13041406524 0020737 0 ustar 00root root 0000000 0000000 /*
* librdkafka - Apache Kafka C library
*
* Copyright (c) 2012, Magnus Edenhill
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* 1. Redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright notice,
* this list of conditions and the following disclaimer in the documentation
* and/or other materials provided with the distribution.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*/
/**
* Apache Kafka consumer & producer example programs
* using the Kafka driver from librdkafka
* (https://github.com/edenhill/librdkafka)
*/
#include
#include
#include
#include
#include
#include
#include
#include
#include
/* Typical include path would be , but this program
* is builtin from within the librdkafka source tree and thus differs. */
#include "rdkafka.h" /* for Kafka driver */
static int run = 1;
static rd_kafka_t *rk;
static int exit_eof = 0;
static int quiet = 0;
static enum {
OUTPUT_HEXDUMP,
OUTPUT_RAW,
} output = OUTPUT_HEXDUMP;
static void stop (int sig) {
run = 0;
fclose(stdin); /* abort fgets() */
}
static void hexdump (FILE *fp, const char *name, const void *ptr, size_t len) {
const char *p = (const char *)ptr;
size_t of = 0;
if (name)
fprintf(fp, "%s hexdump (%zd bytes):\n", name, len);
for (of = 0 ; of < len ; of += 16) {
char hexen[16*3+1];
char charen[16+1];
int hof = 0;
int cof = 0;
int i;
for (i = of ; i < (int)of + 16 && i < (int)len ; i++) {
hof += sprintf(hexen+hof, "%02x ", p[i] & 0xff);
cof += sprintf(charen+cof, "%c",
isprint((int)p[i]) ? p[i] : '.');
}
fprintf(fp, "%08zx: %-48s %-16s\n",
of, hexen, charen);
}
}
/**
* Kafka logger callback (optional)
*/
static void logger (const rd_kafka_t *rk, int level,
const char *fac, const char *buf) {
struct timeval tv;
gettimeofday(&tv, NULL);
fprintf(stderr, "%u.%03u RDKAFKA-%i-%s: %s: %s\n",
(int)tv.tv_sec, (int)(tv.tv_usec / 1000),
level, fac, rk ? rd_kafka_name(rk) : NULL, buf);
}
/**
* Message delivery report callback.
* Called once for each message.
* See rdkafka.h for more information.
*/
static void msg_delivered (rd_kafka_t *rk,
void *payload, size_t len,
int error_code,
void *opaque, void *msg_opaque) {
if (error_code)
fprintf(stderr, "%% Message delivery failed: %s\n",
rd_kafka_err2str(error_code));
else if (!quiet)
fprintf(stderr, "%% Message delivered (%zd bytes): %.*s\n", len,
(int)len, (const char *)payload);
}
/**
* Message delivery report callback using the richer rd_kafka_message_t object.
*/
static void msg_delivered2 (rd_kafka_t *rk,
const rd_kafka_message_t *rkmessage, void *opaque) {
printf("del: %s: offset %"PRId64"\n",
rd_kafka_err2str(rkmessage->err), rkmessage->offset);
if (rkmessage->err)
fprintf(stderr, "%% Message delivery failed: %s\n",
rd_kafka_message_errstr(rkmessage));
else if (!quiet)
fprintf(stderr,
"%% Message delivered (%zd bytes, offset %"PRId64", "
"partition %"PRId32"): %.*s\n",
rkmessage->len, rkmessage->offset,
rkmessage->partition,
(int)rkmessage->len, (const char *)rkmessage->payload);
}
static void msg_consume (rd_kafka_message_t *rkmessage,
void *opaque) {
if (rkmessage->err) {
if (rkmessage->err == RD_KAFKA_RESP_ERR__PARTITION_EOF) {
fprintf(stderr,
"%% Consumer reached end of %s [%"PRId32"] "
"message queue at offset %"PRId64"\n",
rd_kafka_topic_name(rkmessage->rkt),
rkmessage->partition, rkmessage->offset);
if (exit_eof)
run = 0;
return;
}
fprintf(stderr, "%% Consume error for topic \"%s\" [%"PRId32"] "
"offset %"PRId64": %s\n",
rd_kafka_topic_name(rkmessage->rkt),
rkmessage->partition,
rkmessage->offset,
rd_kafka_message_errstr(rkmessage));
if (rkmessage->err == RD_KAFKA_RESP_ERR__UNKNOWN_PARTITION ||
rkmessage->err == RD_KAFKA_RESP_ERR__UNKNOWN_TOPIC)
run = 0;
return;
}
if (!quiet) {
rd_kafka_timestamp_type_t tstype;
int64_t timestamp;
fprintf(stdout, "%% Message (offset %"PRId64", %zd bytes):\n",
rkmessage->offset, rkmessage->len);
timestamp = rd_kafka_message_timestamp(rkmessage, &tstype);
if (tstype != RD_KAFKA_TIMESTAMP_NOT_AVAILABLE) {
const char *tsname = "?";
if (tstype == RD_KAFKA_TIMESTAMP_CREATE_TIME)
tsname = "create time";
else if (tstype == RD_KAFKA_TIMESTAMP_LOG_APPEND_TIME)
tsname = "log append time";
fprintf(stdout, "%% Message timestamp: %s %"PRId64
" (%ds ago)\n",
tsname, timestamp,
!timestamp ? 0 :
(int)time(NULL) - (int)(timestamp/1000));
}
}
if (rkmessage->key_len) {
if (output == OUTPUT_HEXDUMP)
hexdump(stdout, "Message Key",
rkmessage->key, rkmessage->key_len);
else
printf("Key: %.*s\n",
(int)rkmessage->key_len, (char *)rkmessage->key);
}
if (output == OUTPUT_HEXDUMP)
hexdump(stdout, "Message Payload",
rkmessage->payload, rkmessage->len);
else
printf("%.*s\n",
(int)rkmessage->len, (char *)rkmessage->payload);
}
static void metadata_print (const char *topic,
const struct rd_kafka_metadata *metadata) {
int i, j, k;
printf("Metadata for %s (from broker %"PRId32": %s):\n",
topic ? : "all topics",
metadata->orig_broker_id,
metadata->orig_broker_name);
/* Iterate brokers */
printf(" %i brokers:\n", metadata->broker_cnt);
for (i = 0 ; i < metadata->broker_cnt ; i++)
printf(" broker %"PRId32" at %s:%i\n",
metadata->brokers[i].id,
metadata->brokers[i].host,
metadata->brokers[i].port);
/* Iterate topics */
printf(" %i topics:\n", metadata->topic_cnt);
for (i = 0 ; i < metadata->topic_cnt ; i++) {
const struct rd_kafka_metadata_topic *t = &metadata->topics[i];
printf(" topic \"%s\" with %i partitions:",
t->topic,
t->partition_cnt);
if (t->err) {
printf(" %s", rd_kafka_err2str(t->err));
if (t->err == RD_KAFKA_RESP_ERR_LEADER_NOT_AVAILABLE)
printf(" (try again)");
}
printf("\n");
/* Iterate topic's partitions */
for (j = 0 ; j < t->partition_cnt ; j++) {
const struct rd_kafka_metadata_partition *p;
p = &t->partitions[j];
printf(" partition %"PRId32", "
"leader %"PRId32", replicas: ",
p->id, p->leader);
/* Iterate partition's replicas */
for (k = 0 ; k < p->replica_cnt ; k++)
printf("%s%"PRId32,
k > 0 ? ",":"", p->replicas[k]);
/* Iterate partition's ISRs */
printf(", isrs: ");
for (k = 0 ; k < p->isr_cnt ; k++)
printf("%s%"PRId32,
k > 0 ? ",":"", p->isrs[k]);
if (p->err)
printf(", %s\n", rd_kafka_err2str(p->err));
else
printf("\n");
}
}
}
static void sig_usr1 (int sig) {
rd_kafka_dump(stdout, rk);
}
int main (int argc, char **argv) {
rd_kafka_topic_t *rkt;
char *brokers = "localhost:9092";
char mode = 'C';
char *topic = NULL;
int partition = RD_KAFKA_PARTITION_UA;
int opt;
rd_kafka_conf_t *conf;
rd_kafka_topic_conf_t *topic_conf;
char errstr[512];
int64_t start_offset = 0;
int report_offsets = 0;
int do_conf_dump = 0;
char tmp[16];
int64_t seek_offset = 0;
int64_t tmp_offset = 0;
int get_wmarks = 0;
/* Kafka configuration */
conf = rd_kafka_conf_new();
/* Set logger */
rd_kafka_conf_set_log_cb(conf, logger);
/* Quick termination */
snprintf(tmp, sizeof(tmp), "%i", SIGIO);
rd_kafka_conf_set(conf, "internal.termination.signal", tmp, NULL, 0);
/* Topic configuration */
topic_conf = rd_kafka_topic_conf_new();
while ((opt = getopt(argc, argv, "PCLt:p:b:z:qd:o:eX:As:")) != -1) {
switch (opt) {
case 'P':
case 'C':
case 'L':
mode = opt;
break;
case 't':
topic = optarg;
break;
case 'p':
partition = atoi(optarg);
break;
case 'b':
brokers = optarg;
break;
case 'z':
if (rd_kafka_conf_set(conf, "compression.codec",
optarg,
errstr, sizeof(errstr)) !=
RD_KAFKA_CONF_OK) {
fprintf(stderr, "%% %s\n", errstr);
exit(1);
}
break;
case 'o':
case 's':
if (!strcmp(optarg, "end"))
tmp_offset = RD_KAFKA_OFFSET_END;
else if (!strcmp(optarg, "beginning"))
tmp_offset = RD_KAFKA_OFFSET_BEGINNING;
else if (!strcmp(optarg, "stored"))
tmp_offset = RD_KAFKA_OFFSET_STORED;
else if (!strcmp(optarg, "report"))
report_offsets = 1;
else if (!strcmp(optarg, "wmark"))
get_wmarks = 1;
else {
tmp_offset = strtoll(optarg, NULL, 10);
if (tmp_offset < 0)
tmp_offset = RD_KAFKA_OFFSET_TAIL(-tmp_offset);
}
if (opt == 'o')
start_offset = tmp_offset;
else if (opt == 's')
seek_offset = tmp_offset;
break;
case 'e':
exit_eof = 1;
break;
case 'd':
if (rd_kafka_conf_set(conf, "debug", optarg,
errstr, sizeof(errstr)) !=
RD_KAFKA_CONF_OK) {
fprintf(stderr,
"%% Debug configuration failed: "
"%s: %s\n",
errstr, optarg);
exit(1);
}
break;
case 'q':
quiet = 1;
break;
case 'A':
output = OUTPUT_RAW;
break;
case 'X':
{
char *name, *val;
rd_kafka_conf_res_t res;
if (!strcmp(optarg, "list") ||
!strcmp(optarg, "help")) {
rd_kafka_conf_properties_show(stdout);
exit(0);
}
if (!strcmp(optarg, "dump")) {
do_conf_dump = 1;
continue;
}
name = optarg;
if (!(val = strchr(name, '='))) {
char dest[512];
size_t dest_size = sizeof(dest);
/* Return current value for property. */
res = RD_KAFKA_CONF_UNKNOWN;
if (!strncmp(name, "topic.", strlen("topic.")))
res = rd_kafka_topic_conf_get(
topic_conf,
name+strlen("topic."),
dest, &dest_size);
if (res == RD_KAFKA_CONF_UNKNOWN)
res = rd_kafka_conf_get(
conf, name, dest, &dest_size);
if (res == RD_KAFKA_CONF_OK) {
printf("%s = %s\n", name, dest);
exit(0);
} else {
fprintf(stderr,
"%% %s property\n",
res == RD_KAFKA_CONF_UNKNOWN ?
"Unknown" : "Invalid");
exit(1);
}
}
*val = '\0';
val++;
res = RD_KAFKA_CONF_UNKNOWN;
/* Try "topic." prefixed properties on topic
* conf first, and then fall through to global if
* it didnt match a topic configuration property. */
if (!strncmp(name, "topic.", strlen("topic.")))
res = rd_kafka_topic_conf_set(topic_conf,
name+
strlen("topic."),
val,
errstr,
sizeof(errstr));
if (res == RD_KAFKA_CONF_UNKNOWN)
res = rd_kafka_conf_set(conf, name, val,
errstr, sizeof(errstr));
if (res != RD_KAFKA_CONF_OK) {
fprintf(stderr, "%% %s\n", errstr);
exit(1);
}
}
break;
default:
goto usage;
}
}
if (do_conf_dump) {
const char **arr;
size_t cnt;
int pass;
for (pass = 0 ; pass < 2 ; pass++) {
int i;
if (pass == 0) {
arr = rd_kafka_conf_dump(conf, &cnt);
printf("# Global config\n");
} else {
printf("# Topic config\n");
arr = rd_kafka_topic_conf_dump(topic_conf,
&cnt);
}
for (i = 0 ; i < (int)cnt ; i += 2)
printf("%s = %s\n",
arr[i], arr[i+1]);
printf("\n");
rd_kafka_conf_dump_free(arr, cnt);
}
exit(0);
}
if (optind != argc || (mode != 'L' && !topic)) {
usage:
fprintf(stderr,
"Usage: %s -C|-P|-L -t "
"[-p ] [-b ]\n"
"\n"
"librdkafka version %s (0x%08x)\n"
"\n"
" Options:\n"
" -C | -P Consumer or Producer mode\n"
" -L Metadata list mode\n"
" -t Topic to fetch / produce\n"
" -p Partition (random partitioner)\n"
" -b Broker address (localhost:9092)\n"
" -z Enable compression:\n"
" none|gzip|snappy\n"
" -o Start offset (consumer):\n"
" beginning, end, NNNNN or -NNNNN\n"
" wmark returns the current hi&lo "
"watermarks.\n"
" -o report Report message offsets (producer)\n"
" -e Exit consumer when last message\n"
" in partition has been received.\n"
" -d [facs..] Enable debugging contexts:\n"
" %s\n"
" -q Be quiet\n"
" -A Raw payload output (consumer)\n"
" -X Set arbitrary librdkafka "
"configuration property\n"
" Properties prefixed with \"topic.\" "
"will be set on topic object.\n"
" -X list Show full list of supported "
"properties.\n"
" -X Get single property value\n"
"\n"
" In Consumer mode:\n"
" writes fetched messages to stdout\n"
" In Producer mode:\n"
" reads messages from stdin and sends to broker\n"
" In List mode:\n"
" queries broker for metadata information, "
"topic is optional.\n"
"\n"
"\n"
"\n",
argv[0],
rd_kafka_version_str(), rd_kafka_version(),
RD_KAFKA_DEBUG_CONTEXTS);
exit(1);
}
if ((mode == 'C' && !isatty(STDIN_FILENO)) ||
(mode != 'C' && !isatty(STDOUT_FILENO)))
quiet = 1;
signal(SIGINT, stop);
signal(SIGUSR1, sig_usr1);
if (mode == 'P') {
/*
* Producer
*/
char buf[2048];
int sendcnt = 0;
/* Set up a message delivery report callback.
* It will be called once for each message, either on successful
* delivery to broker, or upon failure to deliver to broker. */
/* If offset reporting (-o report) is enabled, use the
* richer dr_msg_cb instead. */
if (report_offsets) {
rd_kafka_topic_conf_set(topic_conf,
"produce.offset.report",
"true", errstr, sizeof(errstr));
rd_kafka_conf_set_dr_msg_cb(conf, msg_delivered2);
} else
rd_kafka_conf_set_dr_cb(conf, msg_delivered);
/* Create Kafka handle */
if (!(rk = rd_kafka_new(RD_KAFKA_PRODUCER, conf,
errstr, sizeof(errstr)))) {
fprintf(stderr,
"%% Failed to create new producer: %s\n",
errstr);
exit(1);
}
rd_kafka_set_log_level(rk, LOG_DEBUG);
/* Add brokers */
if (rd_kafka_brokers_add(rk, brokers) == 0) {
fprintf(stderr, "%% No valid brokers specified\n");
exit(1);
}
/* Create topic */
rkt = rd_kafka_topic_new(rk, topic, topic_conf);
topic_conf = NULL; /* Now owned by topic */
if (!quiet)
fprintf(stderr,
"%% Type stuff and hit enter to send\n");
while (run && fgets(buf, sizeof(buf), stdin)) {
size_t len = strlen(buf);
if (buf[len-1] == '\n')
buf[--len] = '\0';
/* Send/Produce message. */
if (rd_kafka_produce(rkt, partition,
RD_KAFKA_MSG_F_COPY,
/* Payload and length */
buf, len,
/* Optional key and its length */
NULL, 0,
/* Message opaque, provided in
* delivery report callback as
* msg_opaque. */
NULL) == -1) {
fprintf(stderr,
"%% Failed to produce to topic %s "
"partition %i: %s\n",
rd_kafka_topic_name(rkt), partition,
rd_kafka_err2str(rd_kafka_last_error()));
/* Poll to handle delivery reports */
rd_kafka_poll(rk, 0);
continue;
}
if (!quiet)
fprintf(stderr, "%% Sent %zd bytes to topic "
"%s partition %i\n",
len, rd_kafka_topic_name(rkt), partition);
sendcnt++;
/* Poll to handle delivery reports */
rd_kafka_poll(rk, 0);
}
/* Poll to handle delivery reports */
rd_kafka_poll(rk, 0);
/* Wait for messages to be delivered */
while (run && rd_kafka_outq_len(rk) > 0)
rd_kafka_poll(rk, 100);
/* Destroy topic */
rd_kafka_topic_destroy(rkt);
/* Destroy the handle */
rd_kafka_destroy(rk);
} else if (mode == 'C') {
/*
* Consumer
*/
/* Create Kafka handle */
if (!(rk = rd_kafka_new(RD_KAFKA_CONSUMER, conf,
errstr, sizeof(errstr)))) {
fprintf(stderr,
"%% Failed to create new consumer: %s\n",
errstr);
exit(1);
}
rd_kafka_set_log_level(rk, LOG_DEBUG);
/* Add brokers */
if (rd_kafka_brokers_add(rk, brokers) == 0) {
fprintf(stderr, "%% No valid brokers specified\n");
exit(1);
}
if (get_wmarks) {
int64_t lo, hi;
rd_kafka_resp_err_t err;
/* Only query for hi&lo partition watermarks */
if ((err = rd_kafka_query_watermark_offsets(
rk, topic, partition, &lo, &hi, 5000))) {
fprintf(stderr, "%% query_watermark_offsets() "
"failed: %s\n",
rd_kafka_err2str(err));
exit(1);
}
printf("%s [%d]: low - high offsets: "
"%"PRId64" - %"PRId64"\n",
topic, partition, lo, hi);
rd_kafka_destroy(rk);
exit(0);
}
/* Create topic */
rkt = rd_kafka_topic_new(rk, topic, topic_conf);
topic_conf = NULL; /* Now owned by topic */
/* Start consuming */
if (rd_kafka_consume_start(rkt, partition, start_offset) == -1){
rd_kafka_resp_err_t err = rd_kafka_last_error();
fprintf(stderr, "%% Failed to start consuming: %s\n",
rd_kafka_err2str(err));
if (err == RD_KAFKA_RESP_ERR__INVALID_ARG)
fprintf(stderr,
"%% Broker based offset storage "
"requires a group.id, "
"add: -X group.id=yourGroup\n");
exit(1);
}
while (run) {
rd_kafka_message_t *rkmessage;
rd_kafka_resp_err_t err;
/* Poll for errors, etc. */
rd_kafka_poll(rk, 0);
/* Consume single message.
* See rdkafka_performance.c for high speed
* consuming of messages. */
rkmessage = rd_kafka_consume(rkt, partition, 1000);
if (!rkmessage) /* timeout */
continue;
msg_consume(rkmessage, NULL);
/* Return message to rdkafka */
rd_kafka_message_destroy(rkmessage);
if (seek_offset) {
err = rd_kafka_seek(rkt, partition, seek_offset,
2000);
if (err)
printf("Seek failed: %s\n",
rd_kafka_err2str(err));
else
printf("Seeked to %"PRId64"\n",
seek_offset);
seek_offset = 0;
}
}
/* Stop consuming */
rd_kafka_consume_stop(rkt, partition);
while (rd_kafka_outq_len(rk) > 0)
rd_kafka_poll(rk, 10);
/* Destroy topic */
rd_kafka_topic_destroy(rkt);
/* Destroy handle */
rd_kafka_destroy(rk);
} else if (mode == 'L') {
rd_kafka_resp_err_t err = RD_KAFKA_RESP_ERR_NO_ERROR;
/* Create Kafka handle */
if (!(rk = rd_kafka_new(RD_KAFKA_PRODUCER, conf,
errstr, sizeof(errstr)))) {
fprintf(stderr,
"%% Failed to create new producer: %s\n",
errstr);
exit(1);
}
rd_kafka_set_log_level(rk, LOG_DEBUG);
/* Add brokers */
if (rd_kafka_brokers_add(rk, brokers) == 0) {
fprintf(stderr, "%% No valid brokers specified\n");
exit(1);
}
/* Create topic */
if (topic) {
rkt = rd_kafka_topic_new(rk, topic, topic_conf);
topic_conf = NULL; /* Now owned by topic */
} else
rkt = NULL;
while (run) {
const struct rd_kafka_metadata *metadata;
/* Fetch metadata */
err = rd_kafka_metadata(rk, rkt ? 0 : 1, rkt,
&metadata, 5000);
if (err != RD_KAFKA_RESP_ERR_NO_ERROR) {
fprintf(stderr,
"%% Failed to acquire metadata: %s\n",
rd_kafka_err2str(err));
run = 0;
break;
}
metadata_print(topic, metadata);
rd_kafka_metadata_destroy(metadata);
run = 0;
}
/* Destroy topic */
if (rkt)
rd_kafka_topic_destroy(rkt);
/* Destroy the handle */
rd_kafka_destroy(rk);
if (topic_conf)
rd_kafka_topic_conf_destroy(topic_conf);
/* Exit right away, dont wait for background cleanup, we haven't
* done anything important anyway. */
exit(err ? 2 : 0);
}
if (topic_conf)
rd_kafka_topic_conf_destroy(topic_conf);
/* Let background threads clean up and terminate cleanly. */
run = 5;
while (run-- > 0 && rd_kafka_wait_destroyed(1000) == -1)
printf("Waiting for librdkafka to decommission\n");
if (run <= 0)
rd_kafka_dump(stdout, rk);
return 0;
}
librdkafka-0.9.3/examples/rdkafka_example.cpp 0000664 0000000 0000000 00000043527 13041406524 0021267 0 ustar 00root root 0000000 0000000 /*
* librdkafka - Apache Kafka C library
*
* Copyright (c) 2014, Magnus Edenhill
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* 1. Redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright notice,
* this list of conditions and the following disclaimer in the documentation
* and/or other materials provided with the distribution.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*/
/**
* Apache Kafka consumer & producer example programs
* using the Kafka driver from librdkafka
* (https://github.com/edenhill/librdkafka)
*/
#include
#include
#include
#include
#include
#include
#ifdef _MSC_VER
#include "../win32/wingetopt.h"
#elif _AIX
#include
#else
#include
#endif
/*
* Typically include path in a real application would be
* #include
*/
#include "rdkafkacpp.h"
static void metadata_print (const std::string &topic,
const RdKafka::Metadata *metadata) {
std::cout << "Metadata for " << (topic.empty() ? "" : "all topics")
<< "(from broker " << metadata->orig_broker_id()
<< ":" << metadata->orig_broker_name() << std::endl;
/* Iterate brokers */
std::cout << " " << metadata->brokers()->size() << " brokers:" << std::endl;
RdKafka::Metadata::BrokerMetadataIterator ib;
for (ib = metadata->brokers()->begin();
ib != metadata->brokers()->end();
++ib) {
std::cout << " broker " << (*ib)->id() << " at "
<< (*ib)->host() << ":" << (*ib)->port() << std::endl;
}
/* Iterate topics */
std::cout << metadata->topics()->size() << " topics:" << std::endl;
RdKafka::Metadata::TopicMetadataIterator it;
for (it = metadata->topics()->begin();
it != metadata->topics()->end();
++it) {
std::cout << " topic \""<< (*it)->topic() << "\" with "
<< (*it)->partitions()->size() << " partitions:";
if ((*it)->err() != RdKafka::ERR_NO_ERROR) {
std::cout << " " << err2str((*it)->err());
if ((*it)->err() == RdKafka::ERR_LEADER_NOT_AVAILABLE)
std::cout << " (try again)";
}
std::cout << std::endl;
/* Iterate topic's partitions */
RdKafka::TopicMetadata::PartitionMetadataIterator ip;
for (ip = (*it)->partitions()->begin();
ip != (*it)->partitions()->end();
++ip) {
std::cout << " partition " << (*ip)->id()
<< ", leader " << (*ip)->leader()
<< ", replicas: ";
/* Iterate partition's replicas */
RdKafka::PartitionMetadata::ReplicasIterator ir;
for (ir = (*ip)->replicas()->begin();
ir != (*ip)->replicas()->end();
++ir) {
std::cout << (ir == (*ip)->replicas()->begin() ? "":",") << *ir;
}
/* Iterate partition's ISRs */
std::cout << ", isrs: ";
RdKafka::PartitionMetadata::ISRSIterator iis;
for (iis = (*ip)->isrs()->begin(); iis != (*ip)->isrs()->end() ; ++iis)
std::cout << (iis == (*ip)->isrs()->begin() ? "":",") << *iis;
if ((*ip)->err() != RdKafka::ERR_NO_ERROR)
std::cout << ", " << RdKafka::err2str((*ip)->err()) << std::endl;
else
std::cout << std::endl;
}
}
}
static bool run = true;
static bool exit_eof = false;
static void sigterm (int sig) {
run = false;
}
class ExampleDeliveryReportCb : public RdKafka::DeliveryReportCb {
public:
void dr_cb (RdKafka::Message &message) {
std::cout << "Message delivery for (" << message.len() << " bytes): " <<
message.errstr() << std::endl;
if (message.key())
std::cout << "Key: " << *(message.key()) << ";" << std::endl;
}
};
class ExampleEventCb : public RdKafka::EventCb {
public:
void event_cb (RdKafka::Event &event) {
switch (event.type())
{
case RdKafka::Event::EVENT_ERROR:
std::cerr << "ERROR (" << RdKafka::err2str(event.err()) << "): " <<
event.str() << std::endl;
if (event.err() == RdKafka::ERR__ALL_BROKERS_DOWN)
run = false;
break;
case RdKafka::Event::EVENT_STATS:
std::cerr << "\"STATS\": " << event.str() << std::endl;
break;
case RdKafka::Event::EVENT_LOG:
fprintf(stderr, "LOG-%i-%s: %s\n",
event.severity(), event.fac().c_str(), event.str().c_str());
break;
default:
std::cerr << "EVENT " << event.type() <<
" (" << RdKafka::err2str(event.err()) << "): " <<
event.str() << std::endl;
break;
}
}
};
/* Use of this partitioner is pretty pointless since no key is provided
* in the produce() call. */
class MyHashPartitionerCb : public RdKafka::PartitionerCb {
public:
int32_t partitioner_cb (const RdKafka::Topic *topic, const std::string *key,
int32_t partition_cnt, void *msg_opaque) {
return djb_hash(key->c_str(), key->size()) % partition_cnt;
}
private:
static inline unsigned int djb_hash (const char *str, size_t len) {
unsigned int hash = 5381;
for (size_t i = 0 ; i < len ; i++)
hash = ((hash << 5) + hash) + str[i];
return hash;
}
};
void msg_consume(RdKafka::Message* message, void* opaque) {
switch (message->err()) {
case RdKafka::ERR__TIMED_OUT:
break;
case RdKafka::ERR_NO_ERROR:
/* Real message */
std::cout << "Read msg at offset " << message->offset() << std::endl;
if (message->key()) {
std::cout << "Key: " << *message->key() << std::endl;
}
printf("%.*s\n",
static_cast(message->len()),
static_cast(message->payload()));
break;
case RdKafka::ERR__PARTITION_EOF:
/* Last message */
if (exit_eof) {
run = false;
}
break;
case RdKafka::ERR__UNKNOWN_TOPIC:
case RdKafka::ERR__UNKNOWN_PARTITION:
std::cerr << "Consume failed: " << message->errstr() << std::endl;
run = false;
break;
default:
/* Errors */
std::cerr << "Consume failed: " << message->errstr() << std::endl;
run = false;
}
}
class ExampleConsumeCb : public RdKafka::ConsumeCb {
public:
void consume_cb (RdKafka::Message &msg, void *opaque) {
msg_consume(&msg, opaque);
}
};
int main (int argc, char **argv) {
std::string brokers = "localhost";
std::string errstr;
std::string topic_str;
std::string mode;
std::string debug;
int32_t partition = RdKafka::Topic::PARTITION_UA;
int64_t start_offset = RdKafka::Topic::OFFSET_BEGINNING;
bool do_conf_dump = false;
int opt;
MyHashPartitionerCb hash_partitioner;
int use_ccb = 0;
/*
* Create configuration objects
*/
RdKafka::Conf *conf = RdKafka::Conf::create(RdKafka::Conf::CONF_GLOBAL);
RdKafka::Conf *tconf = RdKafka::Conf::create(RdKafka::Conf::CONF_TOPIC);
while ((opt = getopt(argc, argv, "PCLt:p:b:z:qd:o:eX:AM:f:")) != -1) {
switch (opt) {
case 'P':
case 'C':
case 'L':
mode = opt;
break;
case 't':
topic_str = optarg;
break;
case 'p':
if (!strcmp(optarg, "random"))
/* default */;
else if (!strcmp(optarg, "hash")) {
if (tconf->set("partitioner_cb", &hash_partitioner, errstr) !=
RdKafka::Conf::CONF_OK) {
std::cerr << errstr << std::endl;
exit(1);
}
} else
partition = std::atoi(optarg);
break;
case 'b':
brokers = optarg;
break;
case 'z':
if (conf->set("compression.codec", optarg, errstr) !=
RdKafka::Conf::CONF_OK) {
std::cerr << errstr << std::endl;
exit(1);
}
break;
case 'o':
if (!strcmp(optarg, "end"))
start_offset = RdKafka::Topic::OFFSET_END;
else if (!strcmp(optarg, "beginning"))
start_offset = RdKafka::Topic::OFFSET_BEGINNING;
else if (!strcmp(optarg, "stored"))
start_offset = RdKafka::Topic::OFFSET_STORED;
else
start_offset = strtoll(optarg, NULL, 10);
break;
case 'e':
exit_eof = true;
break;
case 'd':
debug = optarg;
break;
case 'M':
if (conf->set("statistics.interval.ms", optarg, errstr) !=
RdKafka::Conf::CONF_OK) {
std::cerr << errstr << std::endl;
exit(1);
}
break;
case 'X':
{
char *name, *val;
if (!strcmp(optarg, "dump")) {
do_conf_dump = true;
continue;
}
name = optarg;
if (!(val = strchr(name, '='))) {
std::cerr << "%% Expected -X property=value, not " <<
name << std::endl;
exit(1);
}
*val = '\0';
val++;
/* Try "topic." prefixed properties on topic
* conf first, and then fall through to global if
* it didnt match a topic configuration property. */
RdKafka::Conf::ConfResult res;
if (!strncmp(name, "topic.", strlen("topic.")))
res = tconf->set(name+strlen("topic."), val, errstr);
else
res = conf->set(name, val, errstr);
if (res != RdKafka::Conf::CONF_OK) {
std::cerr << errstr << std::endl;
exit(1);
}
}
break;
case 'f':
if (!strcmp(optarg, "ccb"))
use_ccb = 1;
else {
std::cerr << "Unknown option: " << optarg << std::endl;
exit(1);
}
break;
default:
goto usage;
}
}
if (mode.empty() || (topic_str.empty() && mode != "L") || optind != argc) {
usage:
std::string features;
conf->get("builtin.features", features);
fprintf(stderr,
"Usage: %s [-C|-P] -t "
"[-p ] [-b ]\n"
"\n"
"librdkafka version %s (0x%08x, builtin.features \"%s\")\n"
"\n"
" Options:\n"
" -C | -P Consumer or Producer mode\n"
" -L Metadata list mode\n"
" -t Topic to fetch / produce\n"
" -p Partition (random partitioner)\n"
" -p Use partitioner:\n"
" random (default), hash\n"
" -b Broker address (localhost:9092)\n"
" -z Enable compression:\n"
" none|gzip|snappy\n"
" -o Start offset (consumer)\n"
" -e Exit consumer when last message\n"
" in partition has been received.\n"
" -d [facs..] Enable debugging contexts:\n"
" %s\n"
" -M Enable statistics\n"
" -X Set arbitrary librdkafka "
"configuration property\n"
" Properties prefixed with \"topic.\" "
"will be set on topic object.\n"
" Use '-X list' to see the full list\n"
" of supported properties.\n"
" -f Set option:\n"
" ccb - use consume_callback\n"
"\n"
" In Consumer mode:\n"
" writes fetched messages to stdout\n"
" In Producer mode:\n"
" reads messages from stdin and sends to broker\n"
"\n"
"\n"
"\n",
argv[0],
RdKafka::version_str().c_str(), RdKafka::version(),
features.c_str(),
RdKafka::get_debug_contexts().c_str());
exit(1);
}
/*
* Set configuration properties
*/
conf->set("metadata.broker.list", brokers, errstr);
if (!debug.empty()) {
if (conf->set("debug", debug, errstr) != RdKafka::Conf::CONF_OK) {
std::cerr << errstr << std::endl;
exit(1);
}
}
ExampleEventCb ex_event_cb;
conf->set("event_cb", &ex_event_cb, errstr);
if (do_conf_dump) {
int pass;
for (pass = 0 ; pass < 2 ; pass++) {
std::list *dump;
if (pass == 0) {
dump = conf->dump();
std::cout << "# Global config" << std::endl;
} else {
dump = tconf->dump();
std::cout << "# Topic config" << std::endl;
}
for (std::list::iterator it = dump->begin();
it != dump->end(); ) {
std::cout << *it << " = ";
it++;
std::cout << *it << std::endl;
it++;
}
std::cout << std::endl;
}
exit(0);
}
signal(SIGINT, sigterm);
signal(SIGTERM, sigterm);
if (mode == "P") {
/*
* Producer mode
*/
if(topic_str.empty())
goto usage;
ExampleDeliveryReportCb ex_dr_cb;
/* Set delivery report callback */
conf->set("dr_cb", &ex_dr_cb, errstr);
/*
* Create producer using accumulated global configuration.
*/
RdKafka::Producer *producer = RdKafka::Producer::create(conf, errstr);
if (!producer) {
std::cerr << "Failed to create producer: " << errstr << std::endl;
exit(1);
}
std::cout << "% Created producer " << producer->name() << std::endl;
/*
* Create topic handle.
*/
RdKafka::Topic *topic = RdKafka::Topic::create(producer, topic_str,
tconf, errstr);
if (!topic) {
std::cerr << "Failed to create topic: " << errstr << std::endl;
exit(1);
}
/*
* Read messages from stdin and produce to broker.
*/
for (std::string line; run && std::getline(std::cin, line);) {
if (line.empty()) {
producer->poll(0);
continue;
}
/*
* Produce message
*/
RdKafka::ErrorCode resp =
producer->produce(topic, partition,
RdKafka::Producer::RK_MSG_COPY /* Copy payload */,
const_cast(line.c_str()), line.size(),
NULL, NULL);
if (resp != RdKafka::ERR_NO_ERROR)
std::cerr << "% Produce failed: " <<
RdKafka::err2str(resp) << std::endl;
else
std::cerr << "% Produced message (" << line.size() << " bytes)" <<
std::endl;
producer->poll(0);
}
run = true;
while (run && producer->outq_len() > 0) {
std::cerr << "Waiting for " << producer->outq_len() << std::endl;
producer->poll(1000);
}
delete topic;
delete producer;
} else if (mode == "C") {
/*
* Consumer mode
*/
if(topic_str.empty())
goto usage;
/*
* Create consumer using accumulated global configuration.
*/
RdKafka::Consumer *consumer = RdKafka::Consumer::create(conf, errstr);
if (!consumer) {
std::cerr << "Failed to create consumer: " << errstr << std::endl;
exit(1);
}
std::cout << "% Created consumer " << consumer->name() << std::endl;
/*
* Create topic handle.
*/
RdKafka::Topic *topic = RdKafka::Topic::create(consumer, topic_str,
tconf, errstr);
if (!topic) {
std::cerr << "Failed to create topic: " << errstr << std::endl;
exit(1);
}
/*
* Start consumer for topic+partition at start offset
*/
RdKafka::ErrorCode resp = consumer->start(topic, partition, start_offset);
if (resp != RdKafka::ERR_NO_ERROR) {
std::cerr << "Failed to start consumer: " <<
RdKafka::err2str(resp) << std::endl;
exit(1);
}
ExampleConsumeCb ex_consume_cb;
/*
* Consume messages
*/
while (run) {
if (use_ccb) {
consumer->consume_callback(topic, partition, 1000,
&ex_consume_cb, &use_ccb);
} else {
RdKafka::Message *msg = consumer->consume(topic, partition, 1000);
msg_consume(msg, NULL);
delete msg;
}
consumer->poll(0);
}
/*
* Stop consumer
*/
consumer->stop(topic, partition);
consumer->poll(1000);
delete topic;
delete consumer;
} else {
/* Metadata mode */
/*
* Create producer using accumulated global configuration.
*/
RdKafka::Producer *producer = RdKafka::Producer::create(conf, errstr);
if (!producer) {
std::cerr << "Failed to create producer: " << errstr << std::endl;
exit(1);
}
std::cout << "% Created producer " << producer->name() << std::endl;
/*
* Create topic handle.
*/
RdKafka::Topic *topic = NULL;
if(!topic_str.empty()) {
topic = RdKafka::Topic::create(producer, topic_str, tconf, errstr);
if (!topic) {
std::cerr << "Failed to create topic: " << errstr << std::endl;
exit(1);
}
}
while (run) {
class RdKafka::Metadata *metadata;
/* Fetch metadata */
RdKafka::ErrorCode err = producer->metadata(topic!=NULL, topic,
&metadata, 5000);
if (err != RdKafka::ERR_NO_ERROR) {
std::cerr << "%% Failed to acquire metadata: "
<< RdKafka::err2str(err) << std::endl;
run = 0;
break;
}
metadata_print(topic_str, metadata);
delete metadata;
run = 0;
}
}
/*
* Wait for RdKafka to decommission.
* This is not strictly needed (when check outq_len() above), but
* allows RdKafka to clean up all its resources before the application
* exits so that memory profilers such as valgrind wont complain about
* memory leaks.
*/
RdKafka::wait_destroyed(5000);
return 0;
}
librdkafka-0.9.3/examples/rdkafka_performance.c 0000664 0000000 0000000 00000126626 13041406524 0021577 0 ustar 00root root 0000000 0000000 /*
* librdkafka - Apache Kafka C library
*
* Copyright (c) 2012, Magnus Edenhill
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* 1. Redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright notice,
* this list of conditions and the following disclaimer in the documentation
* and/or other materials provided with the distribution.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*/
/**
* Apache Kafka consumer & producer performance tester
* using the Kafka driver from librdkafka
* (https://github.com/edenhill/librdkafka)
*/
#define _GNU_SOURCE /* for strndup() */
#include
#include
#include
#include
/* Typical include path would be , but this program
* is built from within the librdkafka source tree and thus differs. */
#include "rdkafka.h" /* for Kafka driver */
/* Do not include these defines from your program, they will not be
* provided by librdkafka. */
#include "rd.h"
#include "rdtime.h"
static int run = 1;
static int forever = 1;
static rd_ts_t dispintvl = 1000;
static int do_seq = 0;
static int exit_after = 0;
static int exit_eof = 0;
static FILE *stats_fp;
static int dr_disp_div;
static int verbosity = 1;
static int latency_mode = 0;
static int report_offset = 0;
static FILE *latency_fp = NULL;
static int msgcnt = -1;
static int incremental_mode = 0;
static int partition_cnt = 0;
static int eof_cnt = 0;
static int with_dr = 1;
static void stop (int sig) {
if (!run)
exit(0);
run = 0;
}
static long int msgs_wait_cnt = 0;
static long int msgs_wait_produce_cnt = 0;
static rd_ts_t t_end;
static rd_kafka_t *global_rk;
struct avg {
int64_t val;
int cnt;
uint64_t ts_start;
};
static struct {
rd_ts_t t_start;
rd_ts_t t_end;
rd_ts_t t_end_send;
uint64_t msgs;
uint64_t msgs_last;
uint64_t msgs_dr_ok;
uint64_t msgs_dr_err;
uint64_t bytes_dr_ok;
uint64_t bytes;
uint64_t bytes_last;
uint64_t tx;
uint64_t tx_err;
uint64_t avg_rtt;
uint64_t offset;
rd_ts_t t_fetch_latency;
rd_ts_t t_last;
rd_ts_t t_enobufs_last;
rd_ts_t t_total;
rd_ts_t latency_last;
rd_ts_t latency_lo;
rd_ts_t latency_hi;
rd_ts_t latency_sum;
int latency_cnt;
int64_t last_offset;
} cnt = {};
/* Returns wall clock time in microseconds */
uint64_t wall_clock (void) {
struct timeval tv;
gettimeofday(&tv, NULL);
return ((uint64_t)tv.tv_sec * 1000000LLU) +
((uint64_t)tv.tv_usec);
}
static void err_cb (rd_kafka_t *rk, int err, const char *reason, void *opaque) {
printf("%% ERROR CALLBACK: %s: %s: %s\n",
rd_kafka_name(rk), rd_kafka_err2str(err), reason);
}
static void throttle_cb (rd_kafka_t *rk, const char *broker_name,
int32_t broker_id, int throttle_time_ms,
void *opaque) {
printf("%% THROTTLED %dms by %s (%"PRId32")\n", throttle_time_ms,
broker_name, broker_id);
}
static void offset_commit_cb (rd_kafka_t *rk, rd_kafka_resp_err_t err,
rd_kafka_topic_partition_list_t *offsets,
void *opaque) {
int i;
if (err || verbosity >= 2)
printf("%% Offset commit of %d partition(s): %s\n",
offsets->cnt, rd_kafka_err2str(err));
for (i = 0 ; i < offsets->cnt ; i++) {
rd_kafka_topic_partition_t *rktpar = &offsets->elems[i];
if (rktpar->err || verbosity >= 2)
printf("%% %s [%"PRId32"] @ %"PRId64": %s\n",
rktpar->topic, rktpar->partition,
rktpar->offset, rd_kafka_err2str(err));
}
}
static void msg_delivered (rd_kafka_t *rk,
const rd_kafka_message_t *rkmessage, void *opaque) {
static rd_ts_t last;
rd_ts_t now = rd_clock();
static int msgs;
msgs++;
msgs_wait_cnt--;
if (rkmessage->err)
cnt.msgs_dr_err++;
else {
cnt.msgs_dr_ok++;
cnt.bytes_dr_ok += rkmessage->len;
}
if ((rkmessage->err &&
(cnt.msgs_dr_err < 50 ||
!(cnt.msgs_dr_err % (dispintvl / 1000)))) ||
!last || msgs_wait_cnt < 5 ||
!(msgs_wait_cnt % dr_disp_div) ||
(now - last) >= dispintvl * 1000 ||
verbosity >= 3) {
if (rkmessage->err && verbosity >= 2)
printf("%% Message delivery failed: %s [%"PRId32"]: "
"%s (%li remain)\n",
rd_kafka_topic_name(rkmessage->rkt),
rkmessage->partition,
rd_kafka_err2str(rkmessage->err),
msgs_wait_cnt);
else if (verbosity > 2)
printf("%% Message delivered (offset %"PRId64"): "
"%li remain\n",
rkmessage->offset, msgs_wait_cnt);
if (verbosity >= 3 && do_seq)
printf(" --> \"%.*s\"\n",
(int)rkmessage->len,
(const char *)rkmessage->payload);
last = now;
}
if (report_offset)
cnt.last_offset = rkmessage->offset;
if (msgs_wait_produce_cnt == 0 && msgs_wait_cnt == 0 && !forever) {
if (verbosity >= 2)
printf("All messages delivered!\n");
t_end = rd_clock();
run = 0;
}
if (exit_after && exit_after <= msgs) {
printf("%% Hard exit after %i messages, as requested\n",
exit_after);
exit(0);
}
}
static void msg_consume (rd_kafka_message_t *rkmessage, void *opaque) {
if (rkmessage->err) {
if (rkmessage->err == RD_KAFKA_RESP_ERR__PARTITION_EOF) {
cnt.offset = rkmessage->offset;
if (verbosity >= 1)
printf("%% Consumer reached end of "
"%s [%"PRId32"] "
"message queue at offset %"PRId64"\n",
rd_kafka_topic_name(rkmessage->rkt),
rkmessage->partition, rkmessage->offset);
if (exit_eof && ++eof_cnt == partition_cnt)
run = 0;
return;
}
printf("%% Consume error for topic \"%s\" [%"PRId32"] "
"offset %"PRId64": %s\n",
rkmessage->rkt ? rd_kafka_topic_name(rkmessage->rkt):"",
rkmessage->partition,
rkmessage->offset,
rd_kafka_message_errstr(rkmessage));
if (rkmessage->err == RD_KAFKA_RESP_ERR__UNKNOWN_PARTITION ||
rkmessage->err == RD_KAFKA_RESP_ERR__UNKNOWN_TOPIC)
run = 0;
cnt.msgs_dr_err++;
return;
}
/* Start measuring from first message received */
if (!cnt.t_start)
cnt.t_start = cnt.t_last = rd_clock();
cnt.offset = rkmessage->offset;
cnt.msgs++;
cnt.bytes += rkmessage->len;
if (verbosity >= 3 ||
(verbosity >= 2 && !(cnt.msgs % 1000000)))
printf("@%"PRId64": %.*s: %.*s\n",
rkmessage->offset,
(int)rkmessage->key_len, (char *)rkmessage->key,
(int)rkmessage->len, (char *)rkmessage->payload);
if (latency_mode) {
int64_t remote_ts, ts;
if (rkmessage->len > 8 &&
!memcmp(rkmessage->payload, "LATENCY:", 8) &&
sscanf(rkmessage->payload, "LATENCY:%"SCNd64,
&remote_ts) == 1) {
ts = wall_clock() - remote_ts;
if (ts > 0 && ts < (1000000 * 60 * 5)) {
if (ts > cnt.latency_hi)
cnt.latency_hi = ts;
if (!cnt.latency_lo || ts < cnt.latency_lo)
cnt.latency_lo = ts;
cnt.latency_last = ts;
cnt.latency_cnt++;
cnt.latency_sum += ts;
if (latency_fp)
fprintf(latency_fp, "%"PRIu64"\n", ts);
} else {
if (verbosity >= 1)
printf("Received latency timestamp is too far off: %"PRId64"us (message offset %"PRId64"): ignored\n",
ts, rkmessage->offset);
}
} else if (verbosity > 1)
printf("not a LATENCY payload: %.*s\n",
(int)rkmessage->len,
(char *)rkmessage->payload);
}
if (msgcnt != -1 && (int)cnt.msgs >= msgcnt)
run = 0;
}
static void rebalance_cb (rd_kafka_t *rk,
rd_kafka_resp_err_t err,
rd_kafka_topic_partition_list_t *partitions,
void *opaque) {
switch (err)
{
case RD_KAFKA_RESP_ERR__ASSIGN_PARTITIONS:
fprintf(stderr,
"%% Group rebalanced: %d partition(s) assigned\n",
partitions->cnt);
eof_cnt = 0;
partition_cnt = partitions->cnt;
rd_kafka_assign(rk, partitions);
break;
case RD_KAFKA_RESP_ERR__REVOKE_PARTITIONS:
fprintf(stderr,
"%% Group rebalanced: %d partition(s) revoked\n",
partitions->cnt);
eof_cnt = 0;
partition_cnt = 0;
rd_kafka_assign(rk, NULL);
break;
default:
break;
}
}
/**
* Find and extract single value from a two-level search.
* First find 'field1', then find 'field2' and extract its value.
* Returns 0 on miss else the value.
*/
static uint64_t json_parse_fields (const char *json, const char **end,
const char *field1, const char *field2) {
const char *t = json;
const char *t2;
int len1 = strlen(field1);
int len2 = strlen(field2);
while ((t2 = strstr(t, field1))) {
uint64_t v;
t = t2;
t += len1;
/* Find field */
if (!(t2 = strstr(t, field2)))
continue;
t2 += len2;
while (isspace((int)*t2))
t2++;
v = strtoull(t2, (char **)&t, 10);
if (t2 == t)
continue;
*end = t;
return v;
}
*end = t + strlen(t);
return 0;
}
/**
* Parse various values from rdkafka stats
*/
static void json_parse_stats (const char *json) {
const char *t;
const int max_avgs = 100; /* max number of brokers to scan for rtt */
uint64_t avg_rtt[max_avgs+1];
int avg_rtt_i = 0;
/* Store totals at end of array */
avg_rtt[max_avgs] = 0;
/* Extract all broker RTTs */
t = json;
while (avg_rtt_i < max_avgs && *t) {
avg_rtt[avg_rtt_i] = json_parse_fields(t, &t,
"\"rtt\":",
"\"avg\":");
/* Skip low RTT values, means no messages are passing */
if (avg_rtt[avg_rtt_i] < 100 /*0.1ms*/)
continue;
avg_rtt[max_avgs] += avg_rtt[avg_rtt_i];
avg_rtt_i++;
}
if (avg_rtt_i > 0)
avg_rtt[max_avgs] /= avg_rtt_i;
cnt.avg_rtt = avg_rtt[max_avgs];
}
static int stats_cb (rd_kafka_t *rk, char *json, size_t json_len,
void *opaque) {
/* Extract values for our own stats */
json_parse_stats(json);
if (stats_fp)
fprintf(stats_fp, "%s\n", json);
return 0;
}
#define _OTYPE_TAB 0x1 /* tabular format */
#define _OTYPE_SUMMARY 0x2 /* summary format */
#define _OTYPE_FORCE 0x4 /* force output regardless of interval timing */
static void print_stats (rd_kafka_t *rk,
int mode, int otype, const char *compression) {
rd_ts_t now = rd_clock();
rd_ts_t t_total;
static int rows_written = 0;
int print_header;
char extra[512];
int extra_of = 0;
*extra = '\0';
#define EXTRA_PRINTF(fmt...) do { \
if (extra_of < sizeof(extra)) \
extra_of += snprintf(extra+extra_of, \
sizeof(extra)-extra_of, fmt); \
} while (0)
if (!(otype & _OTYPE_FORCE) &&
(((otype & _OTYPE_SUMMARY) && verbosity == 0) ||
cnt.t_last + dispintvl > now))
return;
print_header = !rows_written ||(verbosity > 0 && !(rows_written % 20));
if (cnt.t_end_send)
t_total = cnt.t_end_send - cnt.t_start;
else if (cnt.t_end)
t_total = cnt.t_end - cnt.t_start;
else if (cnt.t_start)
t_total = now - cnt.t_start;
else
t_total = 1;
if (mode == 'P') {
if (otype & _OTYPE_TAB) {
#define ROW_START() do {} while (0)
#define COL_HDR(NAME) printf("| %10.10s ", (NAME))
#define COL_PR64(NAME,VAL) printf("| %10"PRIu64" ", (VAL))
#define COL_PRF(NAME,VAL) printf("| %10.2f ", (VAL))
#define ROW_END() do { \
printf("\n"); \
rows_written++; \
} while (0)
if (print_header) {
/* First time, print header */
ROW_START();
COL_HDR("elapsed");
COL_HDR("msgs");
COL_HDR("bytes");
COL_HDR("rtt");
COL_HDR("dr");
COL_HDR("dr_m/s");
COL_HDR("dr_MB/s");
COL_HDR("dr_err");
COL_HDR("tx_err");
COL_HDR("outq");
if (report_offset)
COL_HDR("offset");
ROW_END();
}
ROW_START();
COL_PR64("elapsed", t_total / 1000);
COL_PR64("msgs", cnt.msgs);
COL_PR64("bytes", cnt.bytes);
COL_PR64("rtt", cnt.avg_rtt / 1000);
COL_PR64("dr", cnt.msgs_dr_ok);
COL_PR64("dr_m/s",
((cnt.msgs_dr_ok * 1000000) / t_total));
COL_PRF("dr_MB/s",
(float)((cnt.bytes_dr_ok) / (float)t_total));
COL_PR64("dr_err", cnt.msgs_dr_err);
COL_PR64("tx_err", cnt.tx_err);
COL_PR64("outq",
rk ? (uint64_t)rd_kafka_outq_len(rk) : 0);
if (report_offset)
COL_PR64("offset", (uint64_t)cnt.last_offset);
ROW_END();
}
if (otype & _OTYPE_SUMMARY) {
printf("%% %"PRIu64" messages produced "
"(%"PRIu64" bytes), "
"%"PRIu64" delivered "
"(offset %"PRId64", %"PRIu64" failed) "
"in %"PRIu64"ms: %"PRIu64" msgs/s and "
"%.02f Mb/s, "
"%"PRIu64" produce failures, %i in queue, "
"%s compression\n",
cnt.msgs, cnt.bytes,
cnt.msgs_dr_ok, cnt.last_offset, cnt.msgs_dr_err,
t_total / 1000,
((cnt.msgs_dr_ok * 1000000) / t_total),
(float)((cnt.bytes_dr_ok) / (float)t_total),
cnt.tx_err,
rk ? rd_kafka_outq_len(rk) : 0,
compression);
}
} else {
float latency_avg = 0.0f;
if (latency_mode && cnt.latency_cnt)
latency_avg = (double)cnt.latency_sum /
(float)cnt.latency_cnt;
if (otype & _OTYPE_TAB) {
if (print_header) {
/* First time, print header */
ROW_START();
COL_HDR("elapsed");
COL_HDR("msgs");
COL_HDR("bytes");
COL_HDR("rtt");
COL_HDR("m/s");
COL_HDR("MB/s");
COL_HDR("rx_err");
COL_HDR("offset");
if (latency_mode) {
COL_HDR("lat_curr");
COL_HDR("lat_avg");
COL_HDR("lat_lo");
COL_HDR("lat_hi");
}
ROW_END();
}
ROW_START();
COL_PR64("elapsed", t_total / 1000);
COL_PR64("msgs", cnt.msgs);
COL_PR64("bytes", cnt.bytes);
COL_PR64("rtt", cnt.avg_rtt / 1000);
COL_PR64("m/s",
((cnt.msgs * 1000000) / t_total));
COL_PRF("MB/s",
(float)((cnt.bytes) / (float)t_total));
COL_PR64("rx_err", cnt.msgs_dr_err);
COL_PR64("offset", cnt.offset);
if (latency_mode) {
COL_PRF("lat_curr", cnt.latency_last / 1000.0f);
COL_PRF("lat_avg", latency_avg / 1000.0f);
COL_PRF("lat_lo", cnt.latency_lo / 1000.0f);
COL_PRF("lat_hi", cnt.latency_hi / 1000.0f);
}
ROW_END();
}
if (otype & _OTYPE_SUMMARY) {
if (latency_avg >= 1.0f)
extra_of += snprintf(extra+extra_of,
sizeof(extra)-extra_of,
", latency "
"curr/avg/lo/hi "
"%.2f/%.2f/%.2f/%.2fms",
cnt.latency_last / 1000.0f,
latency_avg / 1000.0f,
cnt.latency_lo / 1000.0f,
cnt.latency_hi / 1000.0f)
;
printf("%% %"PRIu64" messages (%"PRIu64" bytes) "
"consumed in %"PRIu64"ms: %"PRIu64" msgs/s "
"(%.02f Mb/s)"
"%s\n",
cnt.msgs, cnt.bytes,
t_total / 1000,
((cnt.msgs * 1000000) / t_total),
(float)((cnt.bytes) / (float)t_total),
extra);
}
if (incremental_mode && now > cnt.t_last) {
uint64_t i_msgs = cnt.msgs - cnt.msgs_last;
uint64_t i_bytes = cnt.bytes - cnt.bytes_last;
uint64_t i_time = cnt.t_last ? now - cnt.t_last : 0;
printf("%% INTERVAL: %"PRIu64" messages "
"(%"PRIu64" bytes) "
"consumed in %"PRIu64"ms: %"PRIu64" msgs/s "
"(%.02f Mb/s)"
"%s\n",
i_msgs, i_bytes,
i_time / 1000,
((i_msgs * 1000000) / i_time),
(float)((i_bytes) / (float)i_time),
extra);
}
}
cnt.t_last = now;
cnt.msgs_last = cnt.msgs;
cnt.bytes_last = cnt.bytes;
}
static void sig_usr1 (int sig) {
rd_kafka_dump(stdout, global_rk);
}
int main (int argc, char **argv) {
char *brokers = NULL;
char mode = 'C';
char *topic = NULL;
const char *key = NULL;
int *partitions = NULL;
int opt;
int sendflags = 0;
char *msgpattern = "librdkafka_performance testing!";
int msgsize = strlen(msgpattern);
const char *debug = NULL;
rd_ts_t now;
char errstr[512];
uint64_t seq = 0;
int seed = time(NULL);
rd_kafka_t *rk;
rd_kafka_topic_t *rkt;
rd_kafka_conf_t *conf;
rd_kafka_topic_conf_t *topic_conf;
rd_kafka_queue_t *rkqu = NULL;
const char *compression = "no";
int64_t start_offset = 0;
int batch_size = 0;
int idle = 0;
const char *stats_cmd = NULL;
char *stats_intvlstr = NULL;
char tmp[128];
char *tmp2;
int otype = _OTYPE_SUMMARY;
double dtmp;
int rate_sleep = 0;
rd_kafka_topic_partition_list_t *topics;
/* Kafka configuration */
conf = rd_kafka_conf_new();
rd_kafka_conf_set_error_cb(conf, err_cb);
rd_kafka_conf_set_throttle_cb(conf, throttle_cb);
rd_kafka_conf_set_offset_commit_cb(conf, offset_commit_cb);
/* Quick termination */
snprintf(tmp, sizeof(tmp), "%i", SIGIO);
rd_kafka_conf_set(conf, "internal.termination.signal", tmp, NULL, 0);
/* Producer config */
rd_kafka_conf_set(conf, "queue.buffering.max.messages", "500000",
NULL, 0);
rd_kafka_conf_set(conf, "message.send.max.retries", "3", NULL, 0);
rd_kafka_conf_set(conf, "retry.backoff.ms", "500", NULL, 0);
/* Consumer config */
/* Tell rdkafka to (try to) maintain 1M messages
* in its internal receive buffers. This is to avoid
* application -> rdkafka -> broker per-message ping-pong
* latency.
* The larger the local queue, the higher the performance.
* Try other values with: ... -X queued.min.messages=1000
*/
rd_kafka_conf_set(conf, "queued.min.messages", "1000000", NULL, 0);
rd_kafka_conf_set(conf, "session.timeout.ms", "6000", NULL, 0);
/* Kafka topic configuration */
topic_conf = rd_kafka_topic_conf_new();
rd_kafka_topic_conf_set(topic_conf, "auto.offset.reset", "earliest",
NULL, 0);
topics = rd_kafka_topic_partition_list_new(1);
while ((opt =
getopt(argc, argv,
"PCG:t:p:b:s:k:c:fi:MDd:m:S:x:"
"R:a:z:o:X:B:eT:Y:qvIur:lA:OwN")) != -1) {
switch (opt) {
case 'G':
if (rd_kafka_conf_set(conf, "group.id", optarg,
errstr, sizeof(errstr)) !=
RD_KAFKA_CONF_OK) {
fprintf(stderr, "%% %s\n", errstr);
exit(1);
}
/* FALLTHRU */
case 'P':
case 'C':
mode = opt;
break;
case 't':
rd_kafka_topic_partition_list_add(topics, optarg,
RD_KAFKA_PARTITION_UA);
break;
case 'p':
partition_cnt++;
partitions = realloc(partitions, sizeof(*partitions) * partition_cnt);
partitions[partition_cnt-1] = atoi(optarg);
break;
case 'b':
brokers = optarg;
break;
case 's':
msgsize = atoi(optarg);
break;
case 'k':
key = optarg;
break;
case 'c':
msgcnt = atoi(optarg);
break;
case 'D':
sendflags |= RD_KAFKA_MSG_F_FREE;
break;
case 'i':
dispintvl = atoi(optarg);
break;
case 'm':
msgpattern = optarg;
break;
case 'S':
seq = strtoull(optarg, NULL, 10);
do_seq = 1;
break;
case 'x':
exit_after = atoi(optarg);
break;
case 'R':
seed = atoi(optarg);
break;
case 'a':
if (rd_kafka_topic_conf_set(topic_conf,
"request.required.acks",
optarg,
errstr, sizeof(errstr)) !=
RD_KAFKA_CONF_OK) {
fprintf(stderr, "%% %s\n", errstr);
exit(1);
}
break;
case 'B':
batch_size = atoi(optarg);
break;
case 'z':
if (rd_kafka_conf_set(conf, "compression.codec",
optarg,
errstr, sizeof(errstr)) !=
RD_KAFKA_CONF_OK) {
fprintf(stderr, "%% %s\n", errstr);
exit(1);
}
compression = optarg;
break;
case 'o':
if (!strcmp(optarg, "end"))
start_offset = RD_KAFKA_OFFSET_END;
else if (!strcmp(optarg, "beginning"))
start_offset = RD_KAFKA_OFFSET_BEGINNING;
else if (!strcmp(optarg, "stored"))
start_offset = RD_KAFKA_OFFSET_STORED;
else {
start_offset = strtoll(optarg, NULL, 10);
if (start_offset < 0)
start_offset = RD_KAFKA_OFFSET_TAIL(-start_offset);
}
break;
case 'e':
exit_eof = 1;
break;
case 'd':
debug = optarg;
break;
case 'X':
{
char *name, *val;
rd_kafka_conf_res_t res;
if (!strcmp(optarg, "list") ||
!strcmp(optarg, "help")) {
rd_kafka_conf_properties_show(stdout);
exit(0);
}
name = optarg;
if (!(val = strchr(name, '='))) {
fprintf(stderr, "%% Expected "
"-X property=value, not %s\n", name);
exit(1);
}
*val = '\0';
val++;
res = RD_KAFKA_CONF_UNKNOWN;
/* Try "topic." prefixed properties on topic
* conf first, and then fall through to global if
* it didnt match a topic configuration property. */
if (!strncmp(name, "topic.", strlen("topic.")))
res = rd_kafka_topic_conf_set(topic_conf,
name+
strlen("topic."),
val,
errstr,
sizeof(errstr));
if (res == RD_KAFKA_CONF_UNKNOWN)
res = rd_kafka_conf_set(conf, name, val,
errstr, sizeof(errstr));
if (res != RD_KAFKA_CONF_OK) {
fprintf(stderr, "%% %s\n", errstr);
exit(1);
}
}
break;
case 'T':
stats_intvlstr = optarg;
break;
case 'Y':
stats_cmd = optarg;
break;
case 'q':
verbosity--;
break;
case 'v':
verbosity++;
break;
case 'I':
idle = 1;
break;
case 'u':
otype = _OTYPE_TAB;
verbosity--; /* remove some fluff */
break;
case 'r':
dtmp = strtod(optarg, &tmp2);
if (tmp2 == optarg ||
(dtmp >= -0.001 && dtmp <= 0.001)) {
fprintf(stderr, "%% Invalid rate: %s\n",
optarg);
exit(1);
}
rate_sleep = (int)(1000000.0 / dtmp);
break;
case 'l':
latency_mode = 1;
break;
case 'A':
if (!(latency_fp = fopen(optarg, "w"))) {
fprintf(stderr,
"%% Cant open %s: %s\n",
optarg, strerror(errno));
exit(1);
}
break;
case 'O':
if (rd_kafka_topic_conf_set(topic_conf,
"produce.offset.report",
"true",
errstr, sizeof(errstr)) !=
RD_KAFKA_CONF_OK) {
fprintf(stderr, "%% %s\n", errstr);
exit(1);
}
report_offset = 1;
break;
case 'M':
incremental_mode = 1;
break;
case 'N':
with_dr = 0;
break;
default:
fprintf(stderr, "Unknown option: %c\n", opt);
goto usage;
}
}
if (topics->cnt == 0 || optind != argc) {
if (optind < argc)
fprintf(stderr, "Unknown argument: %s\n", argv[optind]);
usage:
fprintf(stderr,
"Usage: %s [-C|-P] -t "
"[-p ] [-b ] [options..]\n"
"\n"
"librdkafka version %s (0x%08x)\n"
"\n"
" Options:\n"
" -C | -P | Consumer or Producer mode\n"
" -G High-level Kafka Consumer mode\n"
" -t Topic to consume / produce\n"
" -p Partition (defaults to random). "
"Multiple partitions are allowed in -C consumer mode.\n"
" -M Print consumer interval stats\n"
" -b Broker address list (host[:port],..)\n"
" -s Message size (producer)\n"
" -k Message key (producer)\n"
" -c Messages to transmit/receive\n"
" -D Copy/Duplicate data buffer (producer)\n"
" -i Display interval\n"
" -m Message payload pattern\n"
" -S Send a sequence number starting at "
" as payload\n"
" -R Random seed value (defaults to time)\n"
" -a Required acks (producer): "
"-1, 0, 1, >1\n"
" -B Consume batch size (# of msgs)\n"
" -z Enable compression:\n"
" none|gzip|snappy\n"
" -o Start offset (consumer)\n"
" beginning, end, NNNNN or -NNNNN\n"
" -d [facs..] Enable debugging contexts:\n"
" %s\n"
" -X Set arbitrary librdkafka "
"configuration property\n"
" Properties prefixed with \"topic.\" "
"will be set on topic object.\n"
" Use '-X list' to see the full list\n"
" of supported properties.\n"
" -T Enable statistics from librdkafka at "
"specified interval (ms)\n"
" -Y Pipe statistics to \n"
" -I Idle: dont produce any messages\n"
" -q Decrease verbosity\n"
" -v Increase verbosity (default 1)\n"
" -u Output stats in table format\n"
" -r Producer msg/s limit\n"
" -l Latency measurement.\n"
" Needs two matching instances, one\n"
" consumer and one producer, both\n"
" running with the -l switch.\n"
" -A Write per-message latency stats to "
". Requires -l\n"
" -O Report produced offset (producer)\n"
" -N No delivery reports (producer)\n"
"\n"
" In Consumer mode:\n"
" consumes messages and prints thruput\n"
" If -B <..> is supplied the batch consumer\n"
" mode is used, else the callback mode is used.\n"
"\n"
" In Producer mode:\n"
" writes messages of size -s <..> and prints thruput\n"
"\n",
argv[0],
rd_kafka_version_str(), rd_kafka_version(),
RD_KAFKA_DEBUG_CONTEXTS);
exit(1);
}
dispintvl *= 1000; /* us */
if (verbosity > 1)
printf("%% Using random seed %i, verbosity level %i\n",
seed, verbosity);
srand(seed);
signal(SIGINT, stop);
signal(SIGUSR1, sig_usr1);
if (debug &&
rd_kafka_conf_set(conf, "debug", debug, errstr, sizeof(errstr)) !=
RD_KAFKA_CONF_OK) {
printf("%% Debug configuration failed: %s: %s\n",
errstr, debug);
exit(1);
}
/* Always enable stats (for RTT extraction), and if user supplied
* the -T option we let her take part of the stats aswell. */
rd_kafka_conf_set_stats_cb(conf, stats_cb);
if (!stats_intvlstr) {
/* if no user-desired stats, adjust stats interval
* to the display interval. */
snprintf(tmp, sizeof(tmp), "%"PRId64, dispintvl / 1000);
}
if (rd_kafka_conf_set(conf, "statistics.interval.ms",
stats_intvlstr ? stats_intvlstr : tmp,
errstr, sizeof(errstr)) !=
RD_KAFKA_CONF_OK) {
fprintf(stderr, "%% %s\n", errstr);
exit(1);
}
if (latency_mode)
do_seq = 0;
if (stats_intvlstr) {
/* User enabled stats (-T) */
if (stats_cmd) {
if (!(stats_fp = popen(stats_cmd, "we"))) {
fprintf(stderr,
"%% Failed to start stats command: "
"%s: %s", stats_cmd, strerror(errno));
exit(1);
}
} else
stats_fp = stdout;
}
if (msgcnt != -1)
forever = 0;
topic = topics->elems[0].topic;
if (mode == 'P') {
/*
* Producer
*/
char *sbuf;
char *pbuf;
int outq;
int keylen = key ? strlen(key) : 0;
off_t rof = 0;
size_t plen = strlen(msgpattern);
int partition = partitions ? partitions[0] :
RD_KAFKA_PARTITION_UA;
if (latency_mode) {
msgsize = strlen("LATENCY:") +
strlen("18446744073709551615 ")+1;
sendflags |= RD_KAFKA_MSG_F_COPY;
} else if (do_seq) {
int minlen = strlen("18446744073709551615 ")+1;
if (msgsize < minlen)
msgsize = minlen;
/* Force duplication of payload */
sendflags |= RD_KAFKA_MSG_F_FREE;
}
sbuf = malloc(msgsize);
/* Copy payload content to new buffer */
while (rof < msgsize) {
size_t xlen = RD_MIN((size_t)msgsize-rof, plen);
memcpy(sbuf+rof, msgpattern, xlen);
rof += xlen;
}
if (msgcnt == -1)
printf("%% Sending messages of size %i bytes\n",
msgsize);
else
printf("%% Sending %i messages of size %i bytes\n",
msgcnt, msgsize);
if (with_dr)
rd_kafka_conf_set_dr_msg_cb(conf, msg_delivered);
/* Create Kafka handle */
if (!(rk = rd_kafka_new(RD_KAFKA_PRODUCER, conf,
errstr, sizeof(errstr)))) {
fprintf(stderr,
"%% Failed to create Kafka producer: %s\n",
errstr);
exit(1);
}
global_rk = rk;
if (debug)
rd_kafka_set_log_level(rk, 7);
/* Add broker(s) */
if (brokers && rd_kafka_brokers_add(rk, brokers) < 1) {
fprintf(stderr, "%% No valid brokers specified\n");
exit(1);
}
/* Explicitly create topic to avoid per-msg lookups. */
rkt = rd_kafka_topic_new(rk, topic, topic_conf);
if (rate_sleep && verbosity >= 2)
fprintf(stderr,
"%% Inter message rate limiter sleep %ius\n",
rate_sleep);
dr_disp_div = msgcnt / 50;
if (dr_disp_div == 0)
dr_disp_div = 10;
cnt.t_start = cnt.t_last = rd_clock();
msgs_wait_produce_cnt = msgcnt;
while (run && (msgcnt == -1 || (int)cnt.msgs < msgcnt)) {
/* Send/Produce message. */
if (idle) {
rd_kafka_poll(rk, 1000);
continue;
}
if (latency_mode) {
snprintf(sbuf, msgsize-1,
"LATENCY:%"PRIu64, wall_clock());
} else if (do_seq) {
snprintf(sbuf,
msgsize-1, "%"PRIu64": ", seq);
seq++;
}
if (sendflags & RD_KAFKA_MSG_F_FREE) {
/* Duplicate memory */
pbuf = malloc(msgsize);
memcpy(pbuf, sbuf, msgsize);
} else
pbuf = sbuf;
if (msgsize == 0)
pbuf = NULL;
cnt.tx++;
while (run &&
rd_kafka_produce(rkt, partition,
sendflags, pbuf, msgsize,
key, keylen, NULL) == -1) {
if (errno == ESRCH)
printf("%% No such partition: "
"%"PRId32"\n", partition);
else if ((errno != ENOBUFS && verbosity >= 1) ||
verbosity >= 3)
printf("%% produce error: %s%s\n",
rd_kafka_err2str(
rd_kafka_errno2err(
errno)),
errno == ENOBUFS ?
" (backpressure)":"");
cnt.tx_err++;
if (errno != ENOBUFS) {
run = 0;
break;
}
now = rd_clock();
if (verbosity >= 2 &&
cnt.t_enobufs_last + dispintvl <= now) {
printf("%% Backpressure %i "
"(tx %"PRIu64", "
"txerr %"PRIu64")\n",
rd_kafka_outq_len(rk),
cnt.tx, cnt.tx_err);
cnt.t_enobufs_last = now;
}
/* Poll to handle delivery reports */
rd_kafka_poll(rk, 10);
print_stats(rk, mode, otype, compression);
}
msgs_wait_cnt++;
if (msgs_wait_produce_cnt != -1)
msgs_wait_produce_cnt--;
cnt.msgs++;
cnt.bytes += msgsize;
if (rate_sleep)
usleep(rate_sleep);
/* Must poll to handle delivery reports */
rd_kafka_poll(rk, 0);
print_stats(rk, mode, otype, compression);
}
forever = 0;
if (verbosity >= 2)
printf("%% All messages produced, "
"now waiting for %li deliveries\n",
msgs_wait_cnt);
if (debug)
rd_kafka_dump(stdout, rk);
/* Wait for messages to be delivered */
while (run && rd_kafka_poll(rk, 1000) != -1)
print_stats(rk, mode, otype, compression);
outq = rd_kafka_outq_len(rk);
if (verbosity >= 2)
printf("%% %i messages in outq\n", outq);
cnt.msgs -= outq;
cnt.bytes -= msgsize * outq;
cnt.t_end = t_end;
if (cnt.tx_err > 0)
printf("%% %"PRIu64" backpressures for %"PRIu64
" produce calls: %.3f%% backpressure rate\n",
cnt.tx_err, cnt.tx,
((double)cnt.tx_err / (double)cnt.tx) * 100.0);
if (debug)
rd_kafka_dump(stdout, rk);
/* Destroy topic */
rd_kafka_topic_destroy(rkt);
/* Destroy the handle */
rd_kafka_destroy(rk);
global_rk = rk = NULL;
free(sbuf);
} else if (mode == 'C') {
/*
* Consumer
*/
rd_kafka_message_t **rkmessages = NULL;
size_t i = 0;
/* Create Kafka handle */
if (!(rk = rd_kafka_new(RD_KAFKA_CONSUMER, conf,
errstr, sizeof(errstr)))) {
fprintf(stderr,
"%% Failed to create Kafka consumer: %s\n",
errstr);
exit(1);
}
global_rk = rk;
if (debug)
rd_kafka_set_log_level(rk, 7);
/* Add broker(s) */
if (brokers && rd_kafka_brokers_add(rk, brokers) < 1) {
fprintf(stderr, "%% No valid brokers specified\n");
exit(1);
}
/* Create topic to consume from */
rkt = rd_kafka_topic_new(rk, topic, topic_conf);
/* Batch consumer */
if (batch_size)
rkmessages = malloc(sizeof(*rkmessages) * batch_size);
/* Start consuming */
rkqu = rd_kafka_queue_new(rk);
for (i=0 ; i<(size_t)partition_cnt ; ++i) {
const int r = rd_kafka_consume_start_queue(rkt,
partitions[i], start_offset, rkqu);
if (r == -1) {
fprintf(stderr, "%% Error creating queue: %s\n",
rd_kafka_err2str(
rd_kafka_errno2err(errno)));
exit(1);
}
}
while (run && (msgcnt == -1 || msgcnt > (int)cnt.msgs)) {
/* Consume messages.
* A message may either be a real message, or
* an error signaling (if rkmessage->err is set).
*/
uint64_t fetch_latency;
int r;
fetch_latency = rd_clock();
if (batch_size) {
int i;
int partition = partitions ? partitions[0] :
RD_KAFKA_PARTITION_UA;
/* Batch fetch mode */
r = rd_kafka_consume_batch(rkt, partition,
1000,
rkmessages,
batch_size);
if (r != -1) {
for (i = 0 ; i < r ; i++) {
msg_consume(rkmessages[i],
NULL);
rd_kafka_message_destroy(
rkmessages[i]);
}
}
} else {
/* Queue mode */
r = rd_kafka_consume_callback_queue(rkqu, 1000,
msg_consume,
NULL);
}
cnt.t_fetch_latency += rd_clock() - fetch_latency;
if (r == -1)
fprintf(stderr, "%% Error: %s\n",
rd_kafka_err2str(
rd_kafka_errno2err(errno)));
print_stats(rk, mode, otype, compression);
/* Poll to handle stats callbacks */
rd_kafka_poll(rk, 0);
}
cnt.t_end = rd_clock();
/* Stop consuming */
for (i=0 ; i<(size_t)partition_cnt ; ++i) {
int r = rd_kafka_consume_stop(rkt, i);
if (r == -1) {
fprintf(stderr,
"%% Error in consume_stop: %s\n",
rd_kafka_err2str(
rd_kafka_errno2err(errno)));
}
}
rd_kafka_queue_destroy(rkqu);
/* Destroy topic */
rd_kafka_topic_destroy(rkt);
if (batch_size)
free(rkmessages);
/* Destroy the handle */
rd_kafka_destroy(rk);
global_rk = rk = NULL;
} else if (mode == 'G') {
/*
* High-level balanced Consumer
*/
rd_kafka_resp_err_t err;
rd_kafka_conf_set_rebalance_cb(conf, rebalance_cb);
rd_kafka_conf_set_default_topic_conf(conf, topic_conf);
/* Create Kafka handle */
if (!(rk = rd_kafka_new(RD_KAFKA_CONSUMER, conf,
errstr, sizeof(errstr)))) {
fprintf(stderr,
"%% Failed to create Kafka consumer: %s\n",
errstr);
exit(1);
}
/* Forward all events to consumer queue */
rd_kafka_poll_set_consumer(rk);
global_rk = rk;
if (debug)
rd_kafka_set_log_level(rk, 7);
/* Add broker(s) */
if (brokers && rd_kafka_brokers_add(rk, brokers) < 1) {
fprintf(stderr, "%% No valid brokers specified\n");
exit(1);
}
err = rd_kafka_subscribe(rk, topics);
if (err) {
fprintf(stderr, "%% Subscribe failed: %s\n",
rd_kafka_err2str(err));
exit(1);
}
fprintf(stderr, "%% Waiting for group rebalance..\n");
while (run && (msgcnt == -1 || msgcnt > (int)cnt.msgs)) {
/* Consume messages.
* A message may either be a real message, or
* an event (if rkmessage->err is set).
*/
rd_kafka_message_t *rkmessage;
uint64_t fetch_latency;
fetch_latency = rd_clock();
rkmessage = rd_kafka_consumer_poll(rk, 1000);
if (rkmessage) {
msg_consume(rkmessage, NULL);
rd_kafka_message_destroy(rkmessage);
}
cnt.t_fetch_latency += rd_clock() - fetch_latency;
print_stats(rk, mode, otype, compression);
}
cnt.t_end = rd_clock();
err = rd_kafka_consumer_close(rk);
if (err)
fprintf(stderr, "%% Failed to close consumer: %s\n",
rd_kafka_err2str(err));
rd_kafka_destroy(rk);
}
print_stats(NULL, mode, otype|_OTYPE_FORCE, compression);
if (cnt.t_fetch_latency && cnt.msgs)
printf("%% Average application fetch latency: %"PRIu64"us\n",
cnt.t_fetch_latency / cnt.msgs);
if (latency_fp)
fclose(latency_fp);
if (stats_fp) {
pclose(stats_fp);
stats_fp = NULL;
}
if (partitions)
free(partitions);
rd_kafka_topic_partition_list_destroy(topics);
/* Let background threads clean up and terminate cleanly. */
rd_kafka_wait_destroyed(2000);
return 0;
}
librdkafka-0.9.3/examples/rdkafka_zookeeper_example.c 0000664 0000000 0000000 00000045604 13041406524 0023010 0 ustar 00root root 0000000 0000000 /*
* librdkafka - Apache Kafka C library
*
* Copyright (c) 2012, Magnus Edenhill
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* 1. Redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright notice,
* this list of conditions and the following disclaimer in the documentation
* and/or other materials provided with the distribution.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*/
/**
* Apache Kafka consumer & producer example programs
* using the Kafka driver from librdkafka
* (https://github.com/edenhill/librdkafka)
*/
#include
#include
#include
#include
#include
#include
#include
#include
/* Typical include path would be , but this program
* is builtin from within the librdkafka source tree and thus differs. */
#include "rdkafka.h" /* for Kafka driver */
#include
#include
#include
#define BROKER_PATH "/brokers/ids"
static int run = 1;
static rd_kafka_t *rk;
static int exit_eof = 0;
static int quiet = 0;
static enum {
OUTPUT_HEXDUMP,
OUTPUT_RAW,
} output = OUTPUT_HEXDUMP;
static void stop (int sig) {
run = 0;
fclose(stdin); /* abort fgets() */
}
static void hexdump (FILE *fp, const char *name, const void *ptr, size_t len) {
const char *p = (const char *)ptr;
int of = 0;
if (name)
fprintf(fp, "%s hexdump (%zd bytes):\n", name, len);
for (of = 0 ; of < len ; of += 16) {
char hexen[16*3+1];
char charen[16+1];
int hof = 0;
int cof = 0;
int i;
for (i = of ; i < of + 16 && i < len ; i++) {
hof += sprintf(hexen+hof, "%02x ", p[i] & 0xff);
cof += sprintf(charen+cof, "%c",
isprint((int)p[i]) ? p[i] : '.');
}
fprintf(fp, "%08x: %-48s %-16s\n",
of, hexen, charen);
}
}
/**
* Kafka logger callback (optional)
*/
static void logger (const rd_kafka_t *rk, int level,
const char *fac, const char *buf) {
struct timeval tv;
gettimeofday(&tv, NULL);
fprintf(stderr, "%u.%03u RDKAFKA-%i-%s: %s: %s\n",
(int)tv.tv_sec, (int)(tv.tv_usec / 1000),
level, fac, rd_kafka_name(rk), buf);
}
/**
* Message delivery report callback.
* Called once for each message.
* See rdkafka.h for more information.
*/
static void msg_delivered (rd_kafka_t *rk,
void *payload, size_t len,
int error_code,
void *opaque, void *msg_opaque) {
if (error_code)
fprintf(stderr, "%% Message delivery failed: %s\n",
rd_kafka_err2str(error_code));
else if (!quiet)
fprintf(stderr, "%% Message delivered (%zd bytes)\n", len);
}
static void msg_consume (rd_kafka_message_t *rkmessage,
void *opaque) {
if (rkmessage->err) {
if (rkmessage->err == RD_KAFKA_RESP_ERR__PARTITION_EOF) {
fprintf(stderr,
"%% Consumer reached end of %s [%"PRId32"] "
"message queue at offset %"PRId64"\n",
rd_kafka_topic_name(rkmessage->rkt),
rkmessage->partition, rkmessage->offset);
if (exit_eof)
run = 0;
return;
}
fprintf(stderr, "%% Consume error for topic \"%s\" [%"PRId32"] "
"offset %"PRId64": %s\n",
rd_kafka_topic_name(rkmessage->rkt),
rkmessage->partition,
rkmessage->offset,
rd_kafka_message_errstr(rkmessage));
return;
}
if (!quiet)
fprintf(stdout, "%% Message (offset %"PRId64", %zd bytes):\n",
rkmessage->offset, rkmessage->len);
if (rkmessage->key_len) {
if (output == OUTPUT_HEXDUMP)
hexdump(stdout, "Message Key",
rkmessage->key, rkmessage->key_len);
else
printf("Key: %.*s\n",
(int)rkmessage->key_len, (char *)rkmessage->key);
}
if (output == OUTPUT_HEXDUMP)
hexdump(stdout, "Message Payload",
rkmessage->payload, rkmessage->len);
else
printf("%.*s\n",
(int)rkmessage->len, (char *)rkmessage->payload);
}
static void metadata_print (const char *topic,
const struct rd_kafka_metadata *metadata) {
int i, j, k;
printf("Metadata for %s (from broker %"PRId32": %s):\n",
topic ? : "all topics",
metadata->orig_broker_id,
metadata->orig_broker_name);
/* Iterate brokers */
printf(" %i brokers:\n", metadata->broker_cnt);
for (i = 0 ; i < metadata->broker_cnt ; i++)
printf(" broker %"PRId32" at %s:%i\n",
metadata->brokers[i].id,
metadata->brokers[i].host,
metadata->brokers[i].port);
/* Iterate topics */
printf(" %i topics:\n", metadata->topic_cnt);
for (i = 0 ; i < metadata->topic_cnt ; i++) {
const struct rd_kafka_metadata_topic *t = &metadata->topics[i];
printf(" topic \"%s\" with %i partitions:",
t->topic,
t->partition_cnt);
if (t->err) {
printf(" %s", rd_kafka_err2str(t->err));
if (t->err == RD_KAFKA_RESP_ERR_LEADER_NOT_AVAILABLE)
printf(" (try again)");
}
printf("\n");
/* Iterate topic's partitions */
for (j = 0 ; j < t->partition_cnt ; j++) {
const struct rd_kafka_metadata_partition *p;
p = &t->partitions[j];
printf(" partition %"PRId32", "
"leader %"PRId32", replicas: ",
p->id, p->leader);
/* Iterate partition's replicas */
for (k = 0 ; k < p->replica_cnt ; k++)
printf("%s%"PRId32,
k > 0 ? ",":"", p->replicas[k]);
/* Iterate partition's ISRs */
printf(", isrs: ");
for (k = 0 ; k < p->isr_cnt ; k++)
printf("%s%"PRId32,
k > 0 ? ",":"", p->isrs[k]);
if (p->err)
printf(", %s\n", rd_kafka_err2str(p->err));
else
printf("\n");
}
}
}
static void set_brokerlist_from_zookeeper(zhandle_t *zzh, char *brokers)
{
if (zzh)
{
struct String_vector brokerlist;
if (zoo_get_children(zzh, BROKER_PATH, 1, &brokerlist) != ZOK)
{
fprintf(stderr, "No brokers found on path %s\n", BROKER_PATH);
return;
}
int i;
char *brokerptr = brokers;
for (i = 0; i < brokerlist.count; i++)
{
char path[255], cfg[1024];
sprintf(path, "/brokers/ids/%s", brokerlist.data[i]);
int len = sizeof(cfg);
zoo_get(zzh, path, 0, cfg, &len, NULL);
if (len > 0)
{
cfg[len] = '\0';
json_error_t jerror;
json_t *jobj = json_loads(cfg, 0, &jerror);
if (jobj)
{
json_t *jhost = json_object_get(jobj, "host");
json_t *jport = json_object_get(jobj, "port");
if (jhost && jport)
{
const char *host = json_string_value(jhost);
const int port = json_integer_value(jport);
sprintf(brokerptr, "%s:%d", host, port);
brokerptr += strlen(brokerptr);
if (i < brokerlist.count - 1)
{
*brokerptr++ = ',';
}
}
json_decref(jobj);
}
}
}
deallocate_String_vector(&brokerlist);
printf("Found brokers %s\n", brokers);
}
}
static void watcher(zhandle_t *zh, int type, int state, const char *path, void *watcherCtx)
{
char brokers[1024];
if (type == ZOO_CHILD_EVENT && strncmp(path, BROKER_PATH, sizeof(BROKER_PATH) - 1) == 0)
{
brokers[0] = '\0';
set_brokerlist_from_zookeeper(zh, brokers);
if (brokers[0] != '\0' && rk != NULL)
{
rd_kafka_brokers_add(rk, brokers);
rd_kafka_poll(rk, 10);
}
}
}
static zhandle_t* initialize_zookeeper(const char * zookeeper, const int debug)
{
zhandle_t *zh;
if (debug)
{
zoo_set_debug_level(ZOO_LOG_LEVEL_DEBUG);
}
zh = zookeeper_init(zookeeper, watcher, 10000, 0, 0, 0);
if (zh == NULL)
{
fprintf(stderr, "Zookeeper connection not established.");
exit(1);
}
return zh;
}
static void sig_usr1 (int sig) {
rd_kafka_dump(stdout, rk);
}
int main (int argc, char **argv) {
rd_kafka_topic_t *rkt;
char *zookeeper = "localhost:2181";
zhandle_t *zh = NULL;
char brokers[1024];
char mode = 'C';
char *topic = NULL;
int partition = RD_KAFKA_PARTITION_UA;
int opt;
rd_kafka_conf_t *conf;
rd_kafka_topic_conf_t *topic_conf;
char errstr[512];
const char *debug = NULL;
int64_t start_offset = 0;
int do_conf_dump = 0;
memset(brokers, 0, sizeof(brokers));
quiet = !isatty(STDIN_FILENO);
/* Kafka configuration */
conf = rd_kafka_conf_new();
/* Topic configuration */
topic_conf = rd_kafka_topic_conf_new();
while ((opt = getopt(argc, argv, "PCLt:p:k:z:qd:o:eX:A")) != -1) {
switch (opt) {
case 'P':
case 'C':
case 'L':
mode = opt;
break;
case 't':
topic = optarg;
break;
case 'p':
partition = atoi(optarg);
break;
case 'k':
zookeeper = optarg;
break;
case 'z':
if (rd_kafka_conf_set(conf, "compression.codec",
optarg,
errstr, sizeof(errstr)) !=
RD_KAFKA_CONF_OK) {
fprintf(stderr, "%% %s\n", errstr);
exit(1);
}
break;
case 'o':
if (!strcmp(optarg, "end"))
start_offset = RD_KAFKA_OFFSET_END;
else if (!strcmp(optarg, "beginning"))
start_offset = RD_KAFKA_OFFSET_BEGINNING;
else if (!strcmp(optarg, "stored"))
start_offset = RD_KAFKA_OFFSET_STORED;
else
start_offset = strtoll(optarg, NULL, 10);
break;
case 'e':
exit_eof = 1;
break;
case 'd':
debug = optarg;
break;
case 'q':
quiet = 1;
break;
case 'A':
output = OUTPUT_RAW;
break;
case 'X':
{
char *name, *val;
rd_kafka_conf_res_t res;
if (!strcmp(optarg, "list") ||
!strcmp(optarg, "help")) {
rd_kafka_conf_properties_show(stdout);
exit(0);
}
if (!strcmp(optarg, "dump")) {
do_conf_dump = 1;
continue;
}
name = optarg;
if (!(val = strchr(name, '='))) {
fprintf(stderr, "%% Expected "
"-X property=value, not %s\n", name);
exit(1);
}
*val = '\0';
val++;
res = RD_KAFKA_CONF_UNKNOWN;
/* Try "topic." prefixed properties on topic
* conf first, and then fall through to global if
* it didnt match a topic configuration property. */
if (!strncmp(name, "topic.", strlen("topic.")))
res = rd_kafka_topic_conf_set(topic_conf,
name+
strlen("topic."),
val,
errstr,
sizeof(errstr));
if (res == RD_KAFKA_CONF_UNKNOWN)
res = rd_kafka_conf_set(conf, name, val,
errstr, sizeof(errstr));
if (res != RD_KAFKA_CONF_OK) {
fprintf(stderr, "%% %s\n", errstr);
exit(1);
}
}
break;
default:
goto usage;
}
}
if (do_conf_dump) {
const char **arr;
size_t cnt;
int pass;
for (pass = 0 ; pass < 2 ; pass++) {
int i;
if (pass == 0) {
arr = rd_kafka_conf_dump(conf, &cnt);
printf("# Global config\n");
} else {
printf("# Topic config\n");
arr = rd_kafka_topic_conf_dump(topic_conf,
&cnt);
}
for (i = 0 ; i < cnt ; i += 2)
printf("%s = %s\n",
arr[i], arr[i+1]);
printf("\n");
rd_kafka_conf_dump_free(arr, cnt);
}
exit(0);
}
if (optind != argc || (mode != 'L' && !topic)) {
usage:
fprintf(stderr,
"Usage: %s -C|-P|-L -t "
"[-p ] [-b ]\n"
"\n"
"librdkafka version %s (0x%08x)\n"
"\n"
" Options:\n"
" -C | -P Consumer or Producer mode\n"
" -L Metadata list mode\n"
" -t Topic to fetch / produce\n"
" -p Partition (random partitioner)\n"
" -k Zookeeper address (localhost:2181)\n"
" -z Enable compression:\n"
" none|gzip|snappy\n"
" -o Start offset (consumer)\n"
" -e Exit consumer when last message\n"
" in partition has been received.\n"
" -d [facs..] Enable debugging contexts:\n"
" -q Be quiet\n"
" -A Raw payload output (consumer)\n"
" %s\n"
" -X Set arbitrary librdkafka "
"configuration property\n"
" Properties prefixed with \"topic.\" "
"will be set on topic object.\n"
" Use '-X list' to see the full list\n"
" of supported properties.\n"
"\n"
" In Consumer mode:\n"
" writes fetched messages to stdout\n"
" In Producer mode:\n"
" reads messages from stdin and sends to broker\n"
" In List mode:\n"
" queries broker for metadata information, "
"topic is optional.\n"
"\n"
"\n"
"\n",
argv[0],
rd_kafka_version_str(), rd_kafka_version(),
RD_KAFKA_DEBUG_CONTEXTS);
exit(1);
}
signal(SIGINT, stop);
signal(SIGUSR1, sig_usr1);
if (debug &&
rd_kafka_conf_set(conf, "debug", debug, errstr, sizeof(errstr)) !=
RD_KAFKA_CONF_OK) {
fprintf(stderr, "%% Debug configuration failed: %s: %s\n",
errstr, debug);
exit(1);
}
/** Initialize zookeeper */
zh = initialize_zookeeper(zookeeper, debug != NULL);
/* Add brokers */
set_brokerlist_from_zookeeper(zh, brokers);
if (rd_kafka_conf_set(conf, "metadata.broker.list",
brokers, errstr, sizeof(errstr) !=
RD_KAFKA_CONF_OK)) {
fprintf(stderr, "%% Failed to set brokers: %s\n", errstr);
exit(1);
}
if (debug) {
printf("Broker list from zookeeper cluster %s: %s\n", zookeeper, brokers);
}
if (mode == 'P') {
/*
* Producer
*/
char buf[2048];
int sendcnt = 0;
/* Set up a message delivery report callback.
* It will be called once for each message, either on successful
* delivery to broker, or upon failure to deliver to broker. */
rd_kafka_conf_set_dr_cb(conf, msg_delivered);
/* Create Kafka handle */
if (!(rk = rd_kafka_new(RD_KAFKA_PRODUCER, conf,
errstr, sizeof(errstr)))) {
fprintf(stderr,
"%% Failed to create new producer: %s\n",
errstr);
exit(1);
}
/* Set logger */
rd_kafka_set_logger(rk, logger);
rd_kafka_set_log_level(rk, LOG_DEBUG);
/* Create topic */
rkt = rd_kafka_topic_new(rk, topic, topic_conf);
if (!quiet)
fprintf(stderr,
"%% Type stuff and hit enter to send\n");
while (run && fgets(buf, sizeof(buf), stdin)) {
size_t len = strlen(buf);
if (buf[len-1] == '\n')
buf[--len] = '\0';
/* Send/Produce message. */
if (rd_kafka_produce(rkt, partition,
RD_KAFKA_MSG_F_COPY,
/* Payload and length */
buf, len,
/* Optional key and its length */
NULL, 0,
/* Message opaque, provided in
* delivery report callback as
* msg_opaque. */
NULL) == -1) {
fprintf(stderr,
"%% Failed to produce to topic %s "
"partition %i: %s\n",
rd_kafka_topic_name(rkt), partition,
rd_kafka_err2str(
rd_kafka_errno2err(errno)));
/* Poll to handle delivery reports */
rd_kafka_poll(rk, 0);
continue;
}
if (!quiet)
fprintf(stderr, "%% Sent %zd bytes to topic "
"%s partition %i\n",
len, rd_kafka_topic_name(rkt), partition);
sendcnt++;
/* Poll to handle delivery reports */
rd_kafka_poll(rk, 0);
}
/* Poll to handle delivery reports */
rd_kafka_poll(rk, 0);
/* Wait for messages to be delivered */
while (run && rd_kafka_outq_len(rk) > 0)
rd_kafka_poll(rk, 100);
/* Destroy the handle */
rd_kafka_destroy(rk);
} else if (mode == 'C') {
/*
* Consumer
*/
/* Create Kafka handle */
if (!(rk = rd_kafka_new(RD_KAFKA_CONSUMER, conf,
errstr, sizeof(errstr)))) {
fprintf(stderr,
"%% Failed to create new consumer: %s\n",
errstr);
exit(1);
}
/* Set logger */
rd_kafka_set_logger(rk, logger);
rd_kafka_set_log_level(rk, LOG_DEBUG);
/* Create topic */
rkt = rd_kafka_topic_new(rk, topic, topic_conf);
/* Start consuming */
if (rd_kafka_consume_start(rkt, partition, start_offset) == -1){
fprintf(stderr, "%% Failed to start consuming: %s\n",
rd_kafka_err2str(rd_kafka_errno2err(errno)));
exit(1);
}
while (run) {
rd_kafka_message_t *rkmessage;
/* Consume single message.
* See rdkafka_performance.c for high speed
* consuming of messages. */
rkmessage = rd_kafka_consume(rkt, partition, 1000);
if (!rkmessage) /* timeout */
continue;
msg_consume(rkmessage, NULL);
/* Return message to rdkafka */
rd_kafka_message_destroy(rkmessage);
}
/* Stop consuming */
rd_kafka_consume_stop(rkt, partition);
rd_kafka_topic_destroy(rkt);
rd_kafka_destroy(rk);
} else if (mode == 'L') {
rd_kafka_resp_err_t err = RD_KAFKA_RESP_ERR_NO_ERROR;
/* Create Kafka handle */
if (!(rk = rd_kafka_new(RD_KAFKA_PRODUCER, conf,
errstr, sizeof(errstr)))) {
fprintf(stderr,
"%% Failed to create new producer: %s\n",
errstr);
exit(1);
}
/* Set logger */
rd_kafka_set_logger(rk, logger);
rd_kafka_set_log_level(rk, LOG_DEBUG);
/* Create topic */
if (topic)
rkt = rd_kafka_topic_new(rk, topic, topic_conf);
else
rkt = NULL;
while (run) {
const struct rd_kafka_metadata *metadata;
/* Fetch metadata */
err = rd_kafka_metadata(rk, rkt ? 0 : 1, rkt,
&metadata, 5000);
if (err != RD_KAFKA_RESP_ERR_NO_ERROR) {
fprintf(stderr,
"%% Failed to acquire metadata: %s\n",
rd_kafka_err2str(err));
run = 0;
break;
}
metadata_print(topic, metadata);
rd_kafka_metadata_destroy(metadata);
run = 0;
}
/* Destroy the handle */
rd_kafka_destroy(rk);
/* Exit right away, dont wait for background cleanup, we haven't
* done anything important anyway. */
exit(err ? 2 : 0);
}
/* Let background threads clean up and terminate cleanly. */
rd_kafka_wait_destroyed(2000);
/** Free the zookeeper data. */
zookeeper_close(zh);
return 0;
}
librdkafka-0.9.3/lds-gen.py 0000775 0000000 0000000 00000001512 13041406524 0015521 0 ustar 00root root 0000000 0000000 #!/usr/bin/env python
#
#
# Generate linker script to only expose symbols of the public API
#
import sys
import re
if __name__ == '__main__':
funcs = list()
last_line = ''
for line in sys.stdin:
m = re.match(r'^(\S+.*\s+\**)?(rd_kafka_\S+)\s*\(', line)
if m:
sym = m.group(2)
# Ignore static (unused) functions
m2 = re.match(r'(RD_UNUSED|__attribute__\(\(unused\)\))', last_line)
if not m2:
funcs.append(sym)
last_line = ''
else:
last_line = line
print('# Automatically generated by lds-gen.py - DO NOT EDIT')
print('{\n global:')
if len(funcs) == 0:
print(' *;')
else:
for f in sorted(funcs):
print(' %s;' % f)
print('local:\n *;')
print('};')
librdkafka-0.9.3/mainpage.doxy 0000664 0000000 0000000 00000002123 13041406524 0016300 0 ustar 00root root 0000000 0000000 /**
* @mainpage librdkafka documentation
*
* librdkafka is the Apache Kafka C/C++ client library.
*
* @section intro Introduction
*
* For an introduction and manual to librdkafka see INTRODUCTION.md
*
* @section conf Configuration
*
* librdkafka is highly configurable to meet any deployment demands.
* It is usually safe to leave most configuration properties to their default
* values.
*
* See CONFIGURATION.md for the full list of supported configuration properties.
*
* @remark Application developers are recommended to provide a non-hardcoded
* interface to librdkafka's string based name-value configuration
* property interface, allowing users to configure any librdkafka
* property directly without alterations to the application.
* This allows for seamless upgrades where linking to a new version
* of librdkafka automatically provides new configuration
* based features.
*
* @section c_api C API
*
* The C API is documented in rdkafka.h
*
* @section cpp_api C++ API
*
* The C++ API is documented in rdkafkacpp.h
*/
librdkafka-0.9.3/mklove/ 0000775 0000000 0000000 00000000000 13041406524 0015111 5 ustar 00root root 0000000 0000000 librdkafka-0.9.3/mklove/Makefile.base 0000775 0000000 0000000 00000011752 13041406524 0017473 0 ustar 00root root 0000000 0000000 # Base Makefile providing various standard targets
# Part of mklove suite but may be used independently.
MKL_RED?= \033[031m
MKL_GREEN?= \033[032m
MKL_YELLOW?= \033[033m
MKL_BLUE?= \033[034m
MKL_CLR_RESET?= \033[0m
DEPS= $(OBJS:%.o=%.d)
# TOPDIR is "TOPDIR/mklove/../" i.e., TOPDIR.
# We do it with two dir calls instead of /.. to support mklove being symlinked.
MKLOVE_DIR := $(dir $(lastword $(MAKEFILE_LIST)))
TOPDIR = $(MKLOVE_DIR:mklove/=.)
# Convert LIBNAME ("libxyz") to "xyz"
LIBNAME0=$(LIBNAME:lib%=%)
# Silence lousy default ARFLAGS (rv)
ARFLAGS=
ifndef MKL_MAKEFILE_CONFIG
-include $(TOPDIR)/Makefile.config
endif
_UNAME_S := $(shell uname -s)
ifeq ($(_UNAME_S),Darwin)
LIBFILENAME=$(LIBNAME).$(LIBVER).dylib
LIBFILENAMELINK=$(LIBNAME).dylib
else
LIBFILENAME=$(LIBNAME).so.$(LIBVER)
LIBFILENAMELINK=$(LIBNAME).so
endif
INSTALL?= install
INSTALL_PROGRAM?= $(INSTALL)
INSTALL_DATA?= $(INSTALL) -m 644
prefix?= /usr/local
exec_prefix?= $(prefix)
bindir?= $(exec_prefix)/bin
sbindir?= $(exec_prefix)/sbin
libexecdir?= $(exec_prefix)/libexec/ # append PKGNAME on install
datarootdir?= $(prefix)/share
datadir?= $(datarootdir) # append PKGNAME on install
sysconfdir?= $(prefix)/etc
sharedstatedir?=$(prefix)/com
localestatedir?=$(prefix)/var
runstatedir?= $(localestatedir)/run
includedir?= $(prefix)/include
docdir?= $(datarootdir)/doc/$(PKGNAME)
infodir?= $(datarootdir)/info
libdir?= $(prefix)/lib
localedir?= $(datarootdir)/locale
pkgconfigdir?= $(libdir)/pkgconfig
mandir?= $(datarootdir)/man
man1dir?= $(mandir)/man1
man2dir?= $(mandir)/man2
man3dir?= $(mandir)/man3
man4dir?= $(mandir)/man4
man5dir?= $(mandir)/man5
man6dir?= $(mandir)/man6
man7dir?= $(mandir)/man7
man8dir?= $(mandir)/man8
# Checks that mklove is set up and ready for building
mklove-check:
@if [ ! -f "$(TOPDIR)/Makefile.config" ]; then \
printf "$(MKL_RED)$(TOPDIR)/Makefile.config missing: please run ./configure$(MKL_CLR_RESET)\n" ; \
exit 1 ; \
fi
%.o: %.c
$(CC) -MD -MP $(CPPFLAGS) $(CFLAGS) -c $< -o $@
%.o: %.cpp
$(CXX) -MD -MP $(CPPFLAGS) $(CXXFLAGS) -c $< -o $@
lib: $(LIBFILENAME) $(LIBNAME).a $(LIBFILENAMELINK) lib-gen-pkg-config
$(LIBFILENAME): $(OBJS) $(LIBNAME).lds
@printf "$(MKL_YELLOW)Creating shared library $@$(MKL_CLR_RESET)\n"
$(CC) $(LDFLAGS) $(LIB_LDFLAGS) $(OBJS) -o $@ $(LIBS)
$(LIBNAME).a: $(OBJS)
@printf "$(MKL_YELLOW)Creating static library $@$(MKL_CLR_RESET)\n"
$(AR) rcs$(ARFLAGS) $@ $(OBJS)
$(LIBFILENAMELINK): $(LIBFILENAME)
@printf "$(MKL_YELLOW)Creating $@ symlink$(MKL_CLR_RESET)\n"
rm -f "$@" && ln -s "$^" "$@"
# pkg-config .pc file definition
ifeq ($(GEN_PKG_CONFIG),y)
define _PKG_CONFIG_DEF
prefix=$(prefix)
libdir=$(libdir)
includedir=$(includedir)
Name: $(LIBNAME)
Description: $(MKL_APP_DESC_ONELINE)
Version: $(MKL_APP_VERSION)
Cflags: -I$${includedir}
Libs: -L$${libdir} -l$(LIBNAME0)
Libs.private: $(patsubst -L%,,$(LIBS))
endef
export _PKG_CONFIG_DEF
$(LIBNAME0).pc: ../Makefile.config
@printf "$(MKL_YELLOW)Generating pkg-config file $@$(MKL_CLR_RESET)\n"
@echo "$$_PKG_CONFIG_DEF" > $@
lib-gen-pkg-config: $(LIBNAME0).pc
lib-clean-pkg-config:
rm -f $(LIBNAME0).pc
else
lib-gen-pkg-config:
lib-clean-pkg-config:
endif
$(BIN): $(OBJS)
@printf "$(MKL_YELLOW)Creating program $@$(MKL_CLR_RESET)\n"
$(CC) $(CPPFLAGS) $(LDFLAGS) $(OBJS) -o $@ $(LIBS)
file-check:
@printf "$(MKL_YELLOW)Checking $(LIBNAME) integrity$(MKL_CLR_RESET)\n"
@RET=true ; \
for f in $(CHECK_FILES) ; do \
printf "%-30s " $$f ; \
if [ -f "$$f" ]; then \
printf "$(MKL_GREEN)OK$(MKL_CLR_RESET)\n" ; \
else \
printf "$(MKL_RED)MISSING$(MKL_CLR_RESET)\n" ; \
RET=false ; \
fi ; \
done ; \
$$RET
lib-install:
@printf "$(MKL_YELLOW)Install $(LIBNAME) to $$DESTDIR$(prefix)$(MKL_CLR_RESET)\n"
$(INSTALL) -d $$DESTDIR$(includedir)/$(PKGNAME) ; \
$(INSTALL) -d $$DESTDIR$(libdir) ; \
$(INSTALL) $(HDRS) $$DESTDIR$(includedir)/$(PKGNAME) ; \
$(INSTALL) $(LIBNAME).a $$DESTDIR$(libdir) ; \
$(INSTALL) $(LIBFILENAME) $$DESTDIR$(libdir) ; \
[ -f "$(LIBNAME0).pc" ] && ( \
$(INSTALL) -d $$DESTDIR$(pkgconfigdir) ; \
$(INSTALL) -m 0644 $(LIBNAME0).pc $$DESTDIR$(pkgconfigdir) \
) ; \
(cd $$DESTDIR$(libdir) && ln -sf $(LIBFILENAME) $(LIBFILENAMELINK))
lib-uninstall:
@printf "$(MKL_YELLOW)Uninstall $(LIBNAME) from $$DESTDIR$(prefix)$(MKL_CLR_RESET)\n"
for hdr in $(HDRS) ; do \
rm -f $$DESTDIR$(includedir)/$(PKGNAME)/$$hdr ; done
rm -f $$DESTDIR$(libdir)/$(LIBNAME).a
rm -f $$DESTDIR$(libdir)/$(LIBFILENAME)
rm -f $$DESTDIR$(libdir)/$(LIBFILENAMELINK)
rmdir $$DESTDIR$(includedir)/$(PKGNAME) || true
bin-install:
@printf "$(MKL_YELLOW)Install $(BIN) to $$DESTDIR$(prefix)$(MKL_CLR_RESET)\n"
$(INSTALL) -d $$DESTDIR$(bindir) && \
$(INSTALL) $(BIN) $$DESTDIR$(bindir)
bin-uninstall:
@printf "$(MKL_YELLOW)Uninstall $(BIN) from $$DESTDIR$(prefix)$(MKL_CLR_RESET)\n"
rm -f $$DESTDIR$(bindir)/$(BIN)
generic-clean:
rm -f $(OBJS) $(DEPS)
lib-clean: generic-clean lib-clean-pkg-config
rm -f $(LIBNAME)*.a $(LIBFILENAME) $(LIBFILENAMELINK) \
$(LIBNAME).lds
bin-clean: generic-clean
rm -f $(BIN)
librdkafka-0.9.3/mklove/modules/ 0000775 0000000 0000000 00000000000 13041406524 0016561 5 ustar 00root root 0000000 0000000 librdkafka-0.9.3/mklove/modules/configure.atomics 0000664 0000000 0000000 00000007735 13041406524 0022137 0 ustar 00root root 0000000 0000000 #!/bin/bash
#
# Checks for atomic ops:
# compiler builtin (__sync_..) and portable libatomic's (__atomic_..)
# Will also provide abstraction by defining the prefix to use.
#
# Sets:
# HAVE_ATOMICS
# HAVE_ATOMICS_32
# HAVE_ATOMICS_64
# HAVE_ATOMICS_32_ATOMIC __atomic interface
# HAVE_ATOMICS_32_SYNC __sync interface
# HAVE_ATOMICS_64_ATOMIC __atomic interface
# HAVE_ATOMICS_64_SYNC __sync interface
# WITH_LIBATOMIC
# LIBS
#
# ATOMIC_OP(OP1,OP2,PTR,VAL)
# ATOMIC_OP32(OP1,OP2,PTR,VAL)
# ATOMIC_OP64(OP1,OP2,PTR,VAL)
# where op* is 'add,sub,fetch'
# e.g: ATOMIC_OP32(add, fetch, &i, 10)
# becomes __atomic_add_fetch(&i, 10, ..) or
# __sync_add_and_fetch(&i, 10)
#
function checks {
# We prefer the newer __atomic stuff, but 64-bit atomics might
# require linking with -latomic, so we need to perform these tests
# in the proper order:
# __atomic 32
# __atomic 32 -latomic
# __sync 32
#
# __atomic 64
# __atomic 64 -latomic
# __sync 64
local _libs=
local _a32="__atomic_ ## OP1 ## _ ## OP2(PTR, VAL, __ATOMIC_SEQ_CST)"
local _a64="__atomic_ ## OP1 ## _ ## OP2(PTR, VAL, __ATOMIC_SEQ_CST)"
# 32-bit:
# Try fully builtin __atomic
if ! mkl_compile_check __atomic_32 HAVE_ATOMICS_32 cont CC "" \
"
#include
int32_t foo (int32_t i) {
return __atomic_add_fetch(&i, 1, __ATOMIC_SEQ_CST);
}"
then
# Try __atomic with -latomic
if mkl_compile_check --ldflags="-latomic" __atomic_32_lib HAVE_ATOMICS_32 \
cont CC "" \
"
#include
int32_t foo (int32_t i) {
return __atomic_add_fetch(&i, 1, __ATOMIC_SEQ_CST);
}"
then
_libs="-latomic"
mkl_allvar_set "__atomic_32_lib" "HAVE_ATOMICS_32_ATOMIC" "y"
else
# Try __sync interface
if mkl_compile_check __sync_32 HAVE_ATOMICS_32 disable CC "" \
"
#include
int32_t foo (int32_t i) {
return __sync_add_and_fetch(&i, 1);
}"
then
_a32="__sync_ ## OP1 ## _and_ ## OP2(PTR, VAL)"
mkl_allvar_set "__sync_32" "HAVE_ATOMICS_32_SYNC" "y"
else
_a32=""
fi
fi
else
mkl_allvar_set "__atomic_32" "HAVE_ATOMICS_32_ATOMIC" "y"
fi
if [[ ! -z $_a32 ]]; then
mkl_define_set "atomic_32" "ATOMIC_OP32(OP1,OP2,PTR,VAL)" "code:$_a32"
fi
# 64-bit:
# Try fully builtin __atomic
if ! mkl_compile_check __atomic_64 HAVE_ATOMICS_64 cont CC "" \
"
#include
int64_t foo (int64_t i) {
return __atomic_add_fetch(&i, 1, __ATOMIC_SEQ_CST);
}"
then
# Try __atomic with -latomic
if mkl_compile_check --ldflags="-latomic" __atomic_64_lib HAVE_ATOMICS_64 \
cont CC "" \
"
#include
int64_t foo (int64_t i) {
return __atomic_add_fetch(&i, 1, __ATOMIC_SEQ_CST);
}"
then
_libs="-latomic"
mkl_allvar_set "__atomic_64_lib" "HAVE_ATOMICS_64_ATOMIC" "y"
else
# Try __sync interface
if mkl_compile_check __sync_64 HAVE_ATOMICS_64 disable CC "" \
"
#include
int64_t foo (int64_t i) {
return __sync_add_and_fetch(&i, 1);
}"
then
_a64="__sync_ ## OP1 ## _and_ ## OP2 (PTR, VAL)"
mkl_allvar_set "__sync_64" "HAVE_ATOMICS_64_SYNC" "y"
else
_a64=""
fi
fi
else
mkl_allvar_set "__atomic_64" "HAVE_ATOMICS_64_ATOMIC" "y"
fi
if [[ ! -z $_a64 ]]; then
mkl_define_set "atomic_64" "ATOMIC_OP64(OP1,OP2,PTR,VAL)" "code:$_a64"
# Define generic ATOMIC() macro identical to 64-bit atomics"
mkl_define_set "atomic_64" "ATOMIC_OP(OP1,OP2,PTR,VAL)" "code:$_a64"
fi
if [[ ! -z $_libs ]]; then
mkl_mkvar_append LDFLAGS LDFLAGS "-Wl,--as-needed"
mkl_mkvar_append LIBS LIBS "$_libs"
fi
}
librdkafka-0.9.3/mklove/modules/configure.base 0000664 0000000 0000000 00000120154 13041406524 0021401 0 ustar 00root root 0000000 0000000 #!/bin/bash
#
#
# mklove base configure module, implements the mklove configure framework
#
MKL_MODULES="base"
MKL_CACHEVARS=""
MKL_MKVARS=""
MKL_DEFINES=""
MKL_CHECKS=""
MKL_LOAD_STACK=""
MKL_IDNEXT=1
MKL_OUTMK=_mklout.mk
MKL_OUTH=_mklout.h
MKL_OUTDBG=config.log
MKL_GENERATORS="base:mkl_generate_late_vars"
MKL_CLEANERS=""
MKL_FAILS=""
MKL_LATE_VARS=""
MKL_OPTS_SET=""
MKL_RED=""
MKL_GREEN=""
MKL_YELLOW=""
MKL_BLUE=""
MKL_CLR_RESET=""
MKL_NO_DOWNLOAD=0
if [[ -z "$MKL_REPO_URL" ]]; then
MKL_REPO_URL="http://github.com/edenhill/mklove/raw/master"
fi
# Default mklove directory to PWD/mklove
[[ -z "$MKLOVE_DIR" ]] && MKLOVE_DIR=mklove
###########################################################################
#
# Variable types:
# env - Standard environment variables.
# var - mklove runtime variable, cached or not.
# mkvar - Makefile variables, also sets runvar
# define - config.h variables/defines
#
###########################################################################
# Low level variable assignment
# Arguments:
# variable name
# variable value
function mkl_var0_set {
export "$1"="$2"
}
# Sets a runtime variable (only used during configure)
# If cache=1 these variables are cached to config.cache.
# Arguments:
# variable name
# variable value
# [ "cache" ]
function mkl_var_set {
mkl_var0_set "$1" "$2"
if [[ $3 == "cache" ]]; then
if ! mkl_in_list "$MKL_CACHEVARS" "$1" ; then
MKL_CACHEVARS="$MKL_CACHEVARS $1"
fi
fi
}
# Unsets a mkl variable
# Arguments:
# variable name
function mkl_var_unset {
unset $1
}
# Appends to a mkl variable (space delimited)
# Arguments:
# variable name
# variable value
function mkl_var_append {
if [[ -z ${!1} ]]; then
mkl_var_set "$1" "$2"
else
mkl_var0_set "$1" "${!1} $2"
fi
}
# Prepends to a mkl variable (space delimited)
# Arguments:
# variable name
# variable value
function mkl_var_prepend {
if [[ -z ${!1} ]]; then
mkl_var_set "$1" "$2"
else
mkl_var0_set "$1" "$2 ${!1}"
fi
}
# Shift the first word off a variable.
# Arguments:
# variable name
function mkl_var_shift {
local n="${!1}"
mkl_var0_set "$1" "${n#* }"
return 0
}
# Returns the contents of mkl variable
# Arguments:
# variable name
function mkl_var_get {
echo "${!1}"
}
# Set environment variable (runtime)
# These variables are not cached nor written to any of the output files,
# its just simply a helper wrapper for standard envs.
# Arguments:
# varname
# varvalue
function mkl_env_set {
mkl_var0_set "$1" "$2"
}
# Append to environment variable
# Arguments:
# varname
# varvalue
# [ separator (" ") ]
function mkl_env_append {
local sep=" "
if [[ -z ${!1} ]]; then
mkl_env_set "$1" "$2"
else
[ ! -z ${3} ] && sep="$3"
mkl_var0_set "$1" "${!1}${sep}$2"
fi
}
# Set a make variable (Makefile.config)
# Arguments:
# config name
# variable name
# value
function mkl_mkvar_set {
if [[ ! -z $2 ]]; then
mkl_env_set "$2" "$3"
mkl_in_list "$MKL_MKVARS" "$2"|| mkl_env_append MKL_MKVARS $2
fi
}
# Appends to a make variable (Makefile.config)
# Arguments:
# config name
# variable name
# value
function mkl_mkvar_append {
if [[ ! -z $2 ]]; then
mkl_env_append "$2" "$3"
mkl_in_list "$MKL_MKVARS" "$2"|| mkl_env_append MKL_MKVARS $2
fi
}
# Return mkvar variable value
# Arguments:
# variable name
function mkl_mkvar_get {
[[ -z ${!1} ]] && return 1
echo ${!1}
return 0
}
# Defines a config header define (config.h)
# Arguments:
# config name
# define name
# define value (optional, default: 1)
# if value starts with code: then no "" are added
function mkl_define_set {
if [[ -z $2 ]]; then
return 0
fi
local stmt=""
local defid=
if [[ $2 = *\(* ]]; then
# macro
defid="def_${2%%(*}"
else
# define
defid="def_$2"
fi
[[ -z $1 ]] || stmt="// $1\n"
local val="$3"
if [[ -z "$val" ]]; then
val="$(mkl_def $2 1)"
fi
# Define as code, string or integer?
if [[ $val == code:* ]]; then
# Code block, copy verbatim without quotes, strip code: prefix
val=${val#code:}
elif [[ ! ( "$val" =~ ^[0-9]+([lL]?[lL][dDuU]?)?$ || \
"$val" =~ ^0x[0-9a-fA-F]+([lL]?[lL][dDuU]?)?$ ) ]]; then
# String: quote
val="\"$val\""
fi
# else: unquoted integer/hex
stmt="${stmt}#define $2 $val"
mkl_env_set "$defid" "$stmt"
mkl_env_append MKL_DEFINES "$defid"
}
# Sets "all" configuration variables, that is:
# for name set: Makefile variable, config.h define
# Will convert value "y"|"n" to 1|0 for config.h
# Arguments:
# config name
# variable name
# value
function mkl_allvar_set {
mkl_mkvar_set "$1" "$2" "$3"
local val=$3
if [[ $3 = "y" ]]; then
val=1
elif [[ $3 = "n" ]]; then
val=0
fi
mkl_define_set "$1" "$2" "$val"
}
###########################################################################
#
#
# Check failure functionality
#
#
###########################################################################
# Summarize all fatal failures and then exits.
function mkl_fail_summary {
echo "
"
local pkg_cmd=""
local install_pkgs=""
mkl_err "###########################################################"
mkl_err "### Configure failed ###"
mkl_err "###########################################################"
mkl_err "### Accumulated failures: ###"
mkl_err "###########################################################"
local n
for n in $MKL_FAILS ; do
local conf=$(mkl_var_get MKL_FAIL__${n}__conf)
mkl_err " $conf ($(mkl_var_get MKL_FAIL__${n}__define)) $(mkl_meta_get $conf name)"
if mkl_meta_exists $conf desc; then
mkl_err0 " desc: $MKL_YELLOW$(mkl_meta_get $conf desc)$MKL_CLR_RESET"
fi
mkl_err0 " module: $(mkl_var_get MKL_FAIL__${n}__module)"
mkl_err0 " action: $(mkl_var_get MKL_FAIL__${n}__action)"
mkl_err0 " reason:
$(mkl_var_get MKL_FAIL__${n}__reason)
"
# Dig up some metadata to assist the user
case $MKL_DISTRO in
Debian|Ubuntu|*)
local debs=$(mkl_meta_get $conf "deb")
pkg_cmd="sudo apt-get install"
if [[ ${#debs} > 0 ]]; then
install_pkgs="$install_pkgs $debs"
fi
;;
esac
done
if [[ ! -z $install_pkgs ]]; then
mkl_err "###########################################################"
mkl_err "### Installing the following packages might help: ###"
mkl_err "###########################################################"
mkl_err0 "$pkg_cmd $install_pkgs"
mkl_err0 ""
fi
exit 1
}
# Checks if there were failures.
# Returns 0 if there were no failures, else calls failure summary and exits.
function mkl_check_fails {
if [[ ${#MKL_FAILS} = 0 ]]; then
return 0
fi
mkl_fail_summary
}
# A check has failed but we want to carry on (and we should!).
# We fail it all later.
# Arguments:
# config name
# define name
# action
# reason
function mkl_fail {
local n="$(mkl_env_esc "$1")"
mkl_var_set "MKL_FAIL__${n}__conf" "$1"
mkl_var_set "MKL_FAIL__${n}__module" $MKL_MODULE
mkl_var_set "MKL_FAIL__${n}__define" $2
mkl_var_set "MKL_FAIL__${n}__action" "$3"
if [[ -z $(mkl_var_get "MKL_FAIL__${n}__reason") ]]; then
mkl_var_set "MKL_FAIL__${n}__reason" "$4"
else
mkl_var_append "MKL_FAIL__${n}__reason" "
And also:
$4"
fi
mkl_in_list "$MKL_FAILS" "$n" || mkl_var_append MKL_FAILS "$n"
}
# A check failed, handle it
# Arguments:
# config name
# define name
# action (fail|disable|ignore|cont)
# reason
function mkl_check_failed {
# Override action based on require directives, unless the action is
# set to cont (for fallthrough to sub-sequent tests).
local action="$3"
if [[ $3 != "cont" ]]; then
action=$(mkl_meta_get "MOD__$MKL_MODULE" "override_action" $3)
fi
# --fail-fatal option
[[ $MKL_FAILFATAL ]] && action="fail"
mkl_check_done "$1" "$2" "$action" "failed"
mkl_dbg "Check $1 ($2, action $action (originally $3)) failed: $4"
case $action in
fail)
# Check failed fatally, fail everything eventually
mkl_fail "$1" "$2" "$3" "$4$extra"
return 1
;;
disable)
# Check failed, disable
[[ ! -z $2 ]] && mkl_mkvar_set "$1" "$2" "n"
return 1
;;
ignore)
# Check failed but we ignore the results and set it anyway.
[[ ! -z $2 ]] && mkl_define_set "$1" "$2" "1"
[[ ! -z $2 ]] && mkl_mkvar_set "$1" "$2" "y"
return 1
;;
cont)
# Check failed but we ignore the results and do nothing.
return 0
;;
esac
}
###########################################################################
#
#
# Output generators
#
#
###########################################################################
# Generate late variables.
# Late variables are those referenced in command line option defaults
# but then never set by --option.
function mkl_generate_late_vars {
local n
for n in $MKL_LATE_VARS ; do
local func=${n%:*}
local safeopt=${func#opt_}
local val=${n#*:}
if mkl_in_list "$MKL_OPTS_SET" "$safeopt" ; then
# Skip options set explicitly with --option
continue
fi
# Expand variable references "\$foo" by calling eval
# and pass it opt_... function.
$func "$(eval echo $val)"
done
}
# Generate output files.
# Must be called following a succesful configure run.
function mkl_generate {
local mf=
for mf in $MKL_GENERATORS ; do
MKL_MODULE=${mf%:*}
local func=${mf#*:}
$func || exit 1
done
mkl_write_mk "# Automatically generated by $0 $*"
mkl_write_mk "# Config variables"
mkl_write_mk "#"
mkl_write_mk "# Generated by:"
mkl_write_mk "# $MKL_CONFIGURE_ARGS"
mkl_write_mk ""
# This variable is used by Makefile.base to avoid multiple inclusions.
mkl_write_mk "MKL_MAKEFILE_CONFIG=y"
# Export colors to Makefile.config
mkl_write_mk "MKL_RED=\t${MKL_RED}"
mkl_write_mk "MKL_GREEN=\t${MKL_GREEN}"
mkl_write_mk "MKL_YELLOW=\t${MKL_YELLOW}"
mkl_write_mk "MKL_BLUE=\t${MKL_BLUE}"
mkl_write_mk "MKL_CLR_RESET=\t${MKL_CLR_RESET}"
local n=
for n in $MKL_MKVARS ; do
# Some special variables should be prefixable by the caller, so
# define them in the makefile as appends.
local op="="
case $n in
CFLAGS|CPPFLAGS|CXXFLAGS|LDFLAGS|LIBS)
op="+="
;;
esac
mkl_write_mk "$n$op\t${!n}"
done
mkl_write_mk "# End of config variables"
MKL_OUTMK_FINAL=Makefile.config
mv $MKL_OUTMK $MKL_OUTMK_FINAL
echo "Generated $MKL_OUTMK_FINAL"
# Generate config.h
mkl_write_h "// Automatically generated by $0 $*"
mkl_write_h "#pragma once"
for n in $MKL_DEFINES ; do
mkl_write_h "${!n}"
done
MKL_OUTH_FINAL=config.h
mv $MKL_OUTH $MKL_OUTH_FINAL
echo "Generated $MKL_OUTH_FINAL"
}
# Remove file noisily, if it exists
function mkl_rm {
if [[ -f $fname ]]; then
echo "Removing $fname"
rm -f "$fname"
fi
}
# Remove files generated by configure
function mkl_clean {
for fname in Makefile.config config.h config.cache config.log ; do
mkl_rm "$fname"
done
local mf=
for mf in $MKL_CLEANERS ; do
MKL_MODULE=${mf%:*}
local func=${mf#*:}
$func || exit 1
done
}
# Print summary of succesful configure run
function mkl_summary {
echo "
Configuration summary:"
local n=
for n in $MKL_MKVARS ; do
# Skip the boring booleans
if [[ $n == WITH_* || $n == WITHOUT_* || $n == HAVE_* || $n == def_* ]]; then
continue
fi
printf " %-24s %s\n" "$n" "${!n}"
done
}
# Write to mk file
# Argument:
# string ..
function mkl_write_mk {
echo -e "$*" >> $MKL_OUTMK
}
# Write to header file
# Argument:
# string ..
function mkl_write_h {
echo -e "$*" >> $MKL_OUTH
}
###########################################################################
#
#
# Logging and debugging
#
#
###########################################################################
# Debug print
# Only visible on terminal if MKL_DEBUG is set.
# Always written to config.log
# Argument:
# string ..
function mkl_dbg {
if [[ ! -z $MKL_DEBUG ]]; then
echo -e "${MKL_BLUE}DBG:$$: $*${MKL_CLR_RESET}" 1>&2
fi
echo "DBG: $*" >> $MKL_OUTDBG
}
# Error print (with color)
# Always printed to terminal and config.log
# Argument:
# string ..
function mkl_err {
echo -e "${MKL_RED}$*${MKL_CLR_RESET}" 1>&2
echo "$*" >> $MKL_OUTDBG
}
# Same as mkl_err but without coloring
# Argument:
# string ..
function mkl_err0 {
echo -e "$*" 1>&2
echo "$*" >> $MKL_OUTDBG
}
# Standard print
# Always printed to terminal and config.log
# Argument:
# string ..
function mkl_info {
echo -e "$*" 1>&2
echo -e "$*" >> $MKL_OUTDBG
}
###########################################################################
#
#
# Misc helpers
#
#
###########################################################################
# Returns the absolute path (but not necesarily canonical) of the first argument
function mkl_abspath {
echo $1 | sed -e "s|^\([^/]\)|$PWD/\1|"
}
# Returns true (0) if function $1 exists, else false (1)
function mkl_func_exists {
declare -f "$1" > /dev/null
return $?
}
# Rename function.
# Returns 0 on success or 1 if old function (origname) was not defined.
# Arguments:
# origname
# newname
function mkl_func_rename {
if ! mkl_func_exists $1 ; then
return 1
fi
local orig=$(declare -f $1)
local new="$2${orig#$1}"
eval "$new"
unset -f "$1"
return 0
}
# Push module function for later call by mklove.
# The function is renamed to an internal name.
# Arguments:
# list variable name
# module name
# function name
function mkl_func_push {
local newfunc="__mkl__f_${2}_$(( MKL_IDNEXT++ ))"
if mkl_func_rename "$3" "$newfunc" ; then
mkl_var_append "$1" "$2:$newfunc"
fi
}
# Returns value, or the default string if value is empty.
# Arguments:
# value
# default
function mkl_def {
if [[ ! -z $1 ]]; then
echo $1
else
echo $2
fi
}
# Render a string (e.g., evaluate its $varrefs)
# Arguments:
# string
function mkl_render {
if [[ $* == *\$* ]]; then
eval "echo $*"
else
echo "$*"
fi
}
# Escape a string so that it becomes suitable for being an env variable.
# This is a destructive operation and the original string cannot be restored.
function mkl_env_esc {
echo $* | LC_ALL=C sed -e 's/[^a-zA-Z0-9_]/_/g'
}
# Convert arguments to upper case
function mkl_upper {
echo "$*" | tr '[:lower:]' '[:upper:]'
}
# Convert arguments to lower case
function mkl_lower {
echo "$*" | tr '[:upper:]' '[:lower:]'
}
# Checks if element is in list
# Arguments:
# list
# element
function mkl_in_list {
local n
for n in $1 ; do
[[ $n == $2 ]] && return 0
done
return 1
}
###########################################################################
#
#
# Cache functionality
#
#
###########################################################################
# Write cache file
function mkl_cache_write {
[[ ! -z "$MKL_NOCACHE" ]] && return 0
echo "# mklove configure cache file generated at $(date)" > config.cache
for n in $MKL_CACHEVARS ; do
echo "$n=${!n}" >> config.cache
done
echo "Generated config.cache"
}
# Read cache file
function mkl_cache_read {
[[ ! -z "$MKL_NOCACHE" ]] && return 0
[ -f config.cache ] || return 1
echo "using cache file config.cache"
local ORIG_IFS=$IFS
IFS="$IFS="
while read -r n v ; do
[[ -z $n || $n = \#* || -z $v ]] && continue
mkl_var_set $n $v cache
done < config.cache
IFS=$ORIG_IFS
}
###########################################################################
#
#
# Config name meta data
#
#
###########################################################################
# Set metadata for config name
# This metadata is used by mkl in various situations
# Arguments:
# config name
# metadata key
# metadata value (appended)
function mkl_meta_set {
local metaname="mkl__$1__$2"
eval "$metaname=\"\$$metaname $3\""
}
# Returns metadata for config name
# Arguments:
# config name
# metadata key
# default (optional)
function mkl_meta_get {
local metaname="mkl__$1__$2"
if [[ ! -z ${!metaname} ]]; then
echo ${!metaname}
else
echo "$3"
fi
}
# Checks if metadata exists
# Arguments:
# config name
# metadata key
function mkl_meta_exists {
local metaname="mkl__$1__$2"
if [[ ! -z ${!metaname} ]]; then
return 0
else
return 1
fi
}
###########################################################################
#
#
# Check framework
#
#
###########################################################################
# Print that a check is beginning to run
# Returns 0 if a cached result was used (do not continue with your tests),
# else 1.
#
# If the check should not be cachable then specify argument 3 as "no-cache",
# this is useful when a check not only checks but actually sets config
# variables itself (which is not recommended, but desired sometimes).
#
# Arguments:
# [ --verb "verb.." ] (replace "checking for")
# config name
# define name
# action (fail,cont,disable or no-cache)
# [ display name ]
function mkl_check_begin {
local verb="checking for"
if [[ $1 == "--verb" ]]; then
verb="$2"
shift
shift
fi
local name=$(mkl_meta_get $1 name "$4")
[[ -z $name ]] && name="x:$1"
echo -n "$verb $name..."
if [[ $3 != "no-cache" ]]; then
local status=$(mkl_var_get "MKL_STATUS_$1")
# Check cache (from previous run or this one).
# Only used cached value if the cached check succeeded:
# it is more likely that a failed check has been fixed than the other
# way around.
if [[ ! -z $status && ( $status = "ok" ) ]]; then
mkl_check_done "$1" "$2" "$3" $status "cached"
return 0
fi
fi
return 1
}
# Print that a check is done
# Arguments:
# config name
# define name
# action
# status (ok|failed)
# extra-info (optional)
function mkl_check_done {
mkl_var_set "MKL_STATUS_$1" "$4" cache
local extra=""
if [[ $4 = "failed" ]]; then
local clr=$MKL_YELLOW
extra=" ($3)"
case "$3" in
fail)
clr=$MKL_RED
;;
cont)
extra=""
;;
esac
echo -e " $clr$4$MKL_CLR_RESET${extra}"
else
[[ ! -z $2 ]] && mkl_define_set "$1" "$2" "1"
[[ ! -z $2 ]] && mkl_mkvar_set "$1" "$2" "y"
[ ! -z "$5" ] && extra=" ($5)"
echo -e " $MKL_GREEN$4${MKL_CLR_RESET}$extra"
fi
}
# Perform configure check by compiling source snippet
# Arguments:
# [--ldflags="..." ] (appended after "compiler arguments" below)
# config name
# define name
# action (fail|disable)
# compiler (CC|CXX)
# compiler arguments (optional "", example: "-lzookeeper")
# source snippet
function mkl_compile_check {
local ldf=
if [[ $1 == --ldflags=* ]]; then
ldf=${1#*=}
shift
fi
mkl_check_begin "$1" "$2" "$3" "$1 (by compile)" && return $?
local cflags=
if [[ $4 = "CXX" ]]; then
local ext=cpp
cflags="$(mkl_mkvar_get CXXFLAGS)"
else
local ext=c
cflags="$(mkl_mkvar_get CFLAGS)"
fi
local srcfile=$(mktemp _mkltmpXXXXXX)
mv "$srcfile" "${srcfile}.$ext"
srcfile="$srcfile.$ext"
echo "$6" > $srcfile
echo "
int main () { return 0; }
" >> $srcfile
local cmd="${!4} $cflags $(mkl_mkvar_get CPPFLAGS) -Wall -Werror $5 $srcfile -o ${srcfile}.o $ldf $(mkl_mkvar_get LDFLAGS)";
mkl_dbg "Compile check $1 ($2): $cmd"
local output
output=$($cmd 2>&1)
if [[ $? != 0 ]] ; then
mkl_dbg "compile check for $1 ($2) failed: $cmd: $output"
mkl_check_failed "$1" "$2" "$3" "compile check failed:
CC: $4
flags: $5
$cmd:
$output
source: $6"
local ret=1
else
mkl_check_done "$1" "$2" "$3" "ok"
local ret=0
fi
# OSX XCode toolchain creates dSYM directories when -g is set,
# delete them specifically.
rm -rf "$srcfile" "${srcfile}.o" "$srcfile*dSYM"
return $ret
}
# Try to link with a library.
# Arguments:
# config name
# define name
# action (fail|disable)
# linker flags (e.g. "-lpthreads")
function mkl_link_check {
mkl_check_begin "$1" "$2" "$3" "$1 (by linking)" && return $?
local srcfile=$(mktemp _mktmpXXXXXX)
echo "int main () { return 0; }" > $srcfile
local cmd="${CC} $(mkl_mkvar_get LDFLAGS) -c $srcfile -o ${srcfile}.o $4";
mkl_dbg "Link check $1 ($2): $cmd"
local output
output=$($cmd 2>&1)
if [[ $? != 0 ]] ; then
mkl_dbg "link check for $1 ($2) failed: $output"
mkl_check_failed "$1" "$2" "$3" "compile check failed:
$output"
local ret=1
else
mkl_check_done "$1" "$2" "$3" "ok" "$4"
local ret=0
fi
rm -f $srcfile*
return $ret
}
# Tries to figure out if we can use a static library or not.
# Arguments:
# library name (e.g. -lrdkafka)
# compiler flags (optional "", e.g: "-lyajl")
# Returns/outputs:
# New list of compiler flags
function mkl_lib_check_static {
local libname=$1
local libs=$2
local arfile_var=STATIC_LIB_${libname#-l}
# If STATIC_LIB_ specifies an existing .a file we
# use that instead.
if [[ -f ${!arfile_var} ]]; then
libs=$(echo $libs | sed -e "s|$libname|${!arfile_var}|g")
else
libs=$(echo $libs | sed -e "s|$libname|${LDFLAGS_STATIC} $libname ${LDFLAGS_DYNAMIC}|g")
fi
echo $libs
}
# Checks that the specified lib is available through a number of methods.
# compiler flags are automatically appended to "LIBS" mkvar on success.
#
# If STATIC_LIB_ is set to the path of an .a file
# it will be used instead of -l.
#
# Arguments:
# [--static=] (allows static linking (--enable-static) for the
# library provided, e.g.: --static=-lrdkafka "librdkafka"..)
# config name (library name (for pkg-config))
# define name
# action (fail|disable|cont)
# compiler (CC|CXX)
# compiler flags (optional "", e.g: "-lyajl")
# source snippet
function mkl_lib_check {
local staticopt=
if [[ $1 == --static* ]]; then
staticopt=$1
shift
fi
# pkg-config result (0=ok)
local pkg_conf_failed=1
if [[ $WITH_PKGCONFIG == "y" ]]; then
# Let pkg-config populate CFLAGS, et.al.
mkl_pkg_config_check $staticopt "${1}" "" cont
pkg_conf_failed=$?
fi
local libs=""
if [[ $pkg_conf_failed ]]; then
libs="$5"
if [[ $WITH_STATIC_LINKING == y && ! -z $staticopt ]]; then
libs=$(mkl_lib_check_static "${staticopt#*=}" "$libs")
fi
fi
if ! mkl_compile_check "$1" "$2" "$3" "$4" "$libs" "$6"; then
return 1
fi
mkl_mkvar_append "$1" LIBS "$libs"
return 0
}
# Check for library with pkg-config
# Automatically sets CFLAGS and LIBS from pkg-config information.
# Arguments:
# [--static=] (allows static linking (--enable-static) for the
# library provided, e.g.: --static=-lrdkafka "librdkafka"..)
# config name
# define name
# action (fail|disable|ignore)
function mkl_pkg_config_check {
local staticopt=
if [[ $1 == --static* ]]; then
staticopt=$1
shift
fi
local cname="${1}_PKGCONFIG"
mkl_check_begin "$cname" "$2" "no-cache" "$1 (by pkg-config)" && return $?
local cflags=
local cmd="${PKG_CONFIG} --short-errors --cflags $1"
mkl_dbg "pkg-config check $1 ($2): $cmd"
cflags=$($cmd 2>&1)
if [[ $? != 0 ]]; then
mkl_dbg "'$cmd' failed: $cflags"
mkl_check_failed "$cname" "$2" "$3" "'$cmd' failed:
$cflags"
return 1
fi
local libs=
libs=$(${PKG_CONFIG} --short-errors --libs $1 2>&1)
if [[ $? != 0 ]]; then
mkl_dbg "${PKG_CONFIG} --libs $1 failed: $libs"
mkl_check_failed "$cname" "$2" "$3" "pkg-config --libs failed"
return 1
fi
mkl_mkvar_append $1 "CFLAGS" "$cflags"
if [[ $WITH_STATIC_LINKING == y && ! -z $staticopt ]]; then
libs=$(mkl_lib_check_static "${staticopt#*=}" "$libs")
fi
mkl_mkvar_append "$1" LIBS "$libs"
mkl_check_done "$cname" "$2" "$3" "ok"
return 0
}
# Check that a command runs and exits succesfully.
# Arguments:
# config name
# define name (optional, can be empty)
# action
# command
function mkl_command_check {
mkl_check_begin "$1" "$2" "$3" "$1 (by command)" && return $?
local out=
out=$($4 2>&1)
if [[ $? != 0 ]]; then
mkl_dbg "$1: $2: $4 failed: $out"
mkl_check_failed "$1" "$2" "$3" "command '$4' failed:
$out"
return 1
fi
mkl_check_done "$1" "$2" "$3" "ok"
return 0
}
# Check that a program is executable, but will not execute it.
# Arguments:
# config name
# define name (optional, can be empty)
# action
# program name (e.g, objdump)
function mkl_prog_check {
mkl_check_begin --verb "checking executable" "$1" "$2" "$3" "$1" && return $?
local out=
out=$(command -v "$4" 2>&1)
if [[ $? != 0 ]]; then
mkl_dbg "$1: $2: $4 is not executable: $out"
mkl_check_failed "$1" "$2" "$3" "$4 is not executable"
return 1
fi
mkl_check_done "$1" "$2" "$3" "ok"
return 0
}
# Checks that the check for the given config name passed.
# This does not behave like the other checks, if the given config name passed
# its test then nothing is printed. Else the configure will fail.
# Arguments:
# checked config name
function mkl_config_check {
local status=$(mkl_var_get "MKL_STATUS_$1")
[[ $status = "ok" ]] && return 0
mkl_fail $1 "" "fail" "$MKL_MODULE requires $1"
return 1
}
# Checks that all provided config names are set.
# Arguments:
# config name
# define name
# action
# check_config_name1
# check_config_name2..
function mkl_config_check_all {
local cname=
local res="ok"
echo start this now for $1
for cname in ${@:4}; do
local st=$(mkl_var_get "MKL_STATUS_$cname")
[[ $status = "ok" ]] && continue
mkl_fail $1 $2 $3 "depends on $cname"
res="failed"
done
echo "try res $res"
mkl_check_done "$1" "$2" "$3" $res
}
# Check environment variable
# Arguments:
# config name
# define name
# action
# environment variable
function mkl_env_check {
mkl_check_begin "$1" "$2" "$3" "$1 (by env $4)" && return $?
if [[ -z ${!4} ]]; then
mkl_check_failed "$1" "$2" "$3" "environment variable $4 not set"
return 1
fi
mkl_check_done "$1" "$2" "$3" "ok" "${!4}"
return 0
}
# Run all checks
function mkl_checks_run {
# Set up common variables
mkl_allvar_set "" MKL_APP_NAME $(mkl_meta_get description name)
mkl_allvar_set "" MKL_APP_DESC_ONELINE "$(mkl_meta_get description oneline)"
# Call checks functions in dependency order
local mf
for mf in $MKL_CHECKS ; do
MKL_MODULE=${mf%:*}
local func=${mf#*:}
if mkl_func_exists $func ; then
$func
else
mkl_err "Check function $func from $MKL_MODULE disappeared ($mf)"
fi
unset MKL_MODULE
done
}
# Check for color support in terminal.
# If the terminal supports colors, the function will alter
# MKL_RED
# MKL_GREEN
# MKL_YELLOW
# MKL_BLUE
# MKL_CLR_RESET
function mkl_check_terminal_color_support {
local use_color=false
local has_tput=false
if [[ -z ${TERM} ]]; then
# tput and dircolors require $TERM
mkl_dbg "\$TERM is not set! Cannot check for color support in terminal."
return 1
elif hash tput 2>/dev/null; then
has_tput=true
[[ $(tput colors 2>/dev/null) -ge 8 ]] && use_color=true
mkl_dbg "tput reports color support: ${use_color}"
elif hash dircolors 2>/dev/null; then
# Enable color support only on colorful terminals.
# dircolors --print-database uses its own built-in database
# instead of using /etc/DIR_COLORS. Try to use the external file
# first to take advantage of user additions.
local safe_term=${TERM//[^[:alnum:]]/?}
local match_lhs=""
[[ -f ~/.dir_colors ]] && match_lhs="${match_lhs}$(<~/.dir_colors)"
[[ -f /etc/DIR_COLORS ]] && match_lhs="${match_lhs}$(&1)
if [[ $? -ne 0 ]]; then
rm -f "$tmpfile"
mkl_err "Failed to download $modname:"
mkl_err0 $out
return 1
fi
# Move downloaded file into place replacing the old file.
mv "$tmpfile" "$fname" || return 1
# "Return" filename
echo "$fname"
return 0
}
# Load module by name or filename
# Arguments:
# "require"|"try"
# filename
# [ module arguments ]
function mkl_module_load {
local try=$1
shift
local fname=$1
shift
local modname=${fname#*configure.}
local bypath=1
# Check if already loaded
if mkl_in_list "$MKL_MODULES" "$modname"; then
return 0
fi
if [[ $fname = $modname ]]; then
# Module specified by name, find the file.
bypath=0
for fname in configure.$modname \
${MKLOVE_DIR}/modules/configure.$modname ; do
[[ -s $fname ]] && break
done
fi
# Calling module
local cmod=$MKL_MODULE
[[ -z $cmod ]] && cmod="base"
if [[ ! -s $fname ]]; then
# Attempt to download module, if permitted
if [[ $MKL_NO_DOWNLOAD != 0 || $bypath == 1 ]]; then
mkl_err "Module $modname not found at $fname (required by $cmod) and downloads disabled"
if [[ $try = "require" ]]; then
mkl_fail "$modname" "none" "fail" \
"Module $modname not found (required by $cmod) and downloads disabled"
fi
return 1
fi
fname=$(mkl_module_download "$modname")
if [[ $? -ne 0 ]]; then
mkl_err "Module $modname not found (required by $cmod)"
if [[ $try = "require" ]]; then
mkl_fail "$modname" "none" "fail" \
"Module $modname not found (required by $cmod)"
return 1
fi
fi
# Now downloaded, try loading the module again.
mkl_module_load $try "$fname" "$@"
return $?
fi
# Set current module
local save_MKL_MODULE=$MKL_MODULE
MKL_MODULE=$modname
mkl_dbg "Loading module $modname (required by $cmod) from $fname"
# Source module file (positional arguments are available to module)
source $fname
# Restore current module (might be recursive)
MKL_MODULE=$save_MKL_MODULE
# Add module to list of modules
mkl_var_append MKL_MODULES $modname
# Rename module's special functions so we can call them separetely later.
mkl_func_rename "options" "${modname}_options"
mkl_func_push MKL_CHECKS "$modname" "checks"
mkl_func_push MKL_GENERATORS "$modname" "generate"
mkl_func_push MKL_CLEANERS "$modname" "clean"
}
# Require and load module
# Must only be called from module file outside any function.
# Arguments:
# [ --try ] Dont fail if module doesn't exist
# module1
# [ "must" "pass" ]
# [ module arguments ... ]
function mkl_require {
local try="require"
if [[ $1 = "--try" ]]; then
local try="try"
shift
fi
local mod=$1
shift
local override_action=
# Check for cyclic dependencies
if mkl_in_list "$MKL_LOAD_STACK" "$mod"; then
mkl_err "Cyclic dependency detected while loading $mod module:"
local cmod=
local lmod=$mod
for cmod in $MKL_LOAD_STACK ; do
mkl_err " $lmod required by $cmod"
lmod=$cmod
done
mkl_fail base "" fail "Cyclic dependency detected while loading module $mod"
return 1
fi
mkl_var_prepend MKL_LOAD_STACK "$mod"
if [[ "$1 $2" == "must pass" ]]; then
shift
shift
override_action="fail"
fi
if [[ ! -z $override_action ]]; then
mkl_meta_set "MOD__$mod" "override_action" "$override_action"
fi
mkl_module_load $try $mod "$@"
local ret=$?
mkl_var_shift MKL_LOAD_STACK
return $ret
}
###########################################################################
#
#
# Usage options
#
#
###########################################################################
MKL_USAGE="Usage: ./configure [OPTIONS...]
mklove configure script - mklove, not autoconf
Copyright (c) 2014-2015 Magnus Edenhill - https://github.com/edenhill/mklove
"
function mkl_usage {
echo "$MKL_USAGE"
local name=$(mkl_meta_get description name)
if [[ ! -z ${name} ]]; then
echo " $name - $(mkl_meta_get description oneline)
$(mkl_meta_get description copyright)
"
fi
local og
for og in $MKL_USAGE_GROUPS ; do
og="MKL_USAGE_GROUP__$og"
echo "${!og}"
done
echo "Honoured environment variables:
CC, CPP, CXX, CFLAGS, CPPFLAGS, CXXFLAGS, LDFLAGS, LIBS,
LD, NM, OBJDUMP, STRIP, PKG_CONFIG, PKG_CONFIG_PATH,
STATIC_LIB_=.../libname.a
"
}
# Add usage option informative text
# Arguments:
# text
function mkl_usage_info {
MKL_USAGE="$MKL_USAGE
$1"
}
# Add option to usage output
# Arguments:
# option group ("Standard", "Cross-Compilation", etc..)
# variable name
# option ("--foo=feh")
# help
# default (optional)
# assignvalue (optional, default:"y")
# function block (optional)
function mkl_option {
local optgroup=$1
local varname=$2
# Fixed width between option name and help in usage output
local pad=" "
if [[ ${#3} -lt ${#pad} ]]; then
pad=${pad:0:$(expr ${#pad} - ${#3})}
else
pad=""
fi
# Add to usage output
local optgroup_safe=$(mkl_env_esc $optgroup)
if ! mkl_in_list "$MKL_USAGE_GROUPS" "$optgroup_safe" ; then
mkl_env_append MKL_USAGE_GROUPS "$optgroup_safe"
mkl_env_set "MKL_USAGE_GROUP__$optgroup_safe" "$optgroup options:
"
fi
local defstr=""
[[ ! -z $5 ]] && defstr=" [$5]"
mkl_env_append "MKL_USAGE_GROUP__$optgroup_safe" " $3 $pad $4$defstr
"
local optname="${3#--}"
local safeopt=
local optval=""
if [[ $3 == *=* ]]; then
optname="${optname%=*}"
optval="${3#*=}"
fi
safeopt=$(mkl_env_esc $optname)
mkl_meta_set "MKL_OPT_ARGS" "$safeopt" "$optval"
#
# Optional variable scoping by prefix: "env:", "mk:", "def:"
#
local setallvar="mkl_allvar_set ''"
local setmkvar="mkl_mkvar_set ''"
if [[ $varname = env:* ]]; then
# Set environment variable (during configure runtime only)
varname=${varname#*:}
setallvar=mkl_env_set
setmkvar=mkl_env_set
elif [[ $varname = mk:* ]]; then
# Set Makefile.config variable
varname=${varname#*:}
setallvar="mkl_mkvar_append ''"
setmkvar="mkl_mkvar_append ''"
elif [[ $varname = def:* ]]; then
# Set config.h define
varname=${varname#*:}
setallvar="mkl_define_set ''"
setmkvar="mkl_define_set ''"
fi
if [[ ! -z $7 ]]; then
# Function block specified.
eval "function opt_$safeopt { $7 }"
else
# Add default implementation of function simply setting the value.
# Application may override this by redefining the function after calling
# mkl_option.
if [[ $optval = "PATH" ]]; then
# PATH argument: make it an absolute path.
# Only set the make variable (not config.h)
eval "function opt_$safeopt { $setmkvar $varname \"\$(mkl_abspath \$(mkl_render \$1))\"; }"
else
# Standard argument: simply set the value
if [[ -z "$6" ]]; then
eval "function opt_$safeopt { $setallvar $varname \"\$1\"; }"
else
eval "function opt_$safeopt { $setallvar $varname \"$6\"; }"
fi
fi
fi
# If default value is provided and does not start with "$" (variable ref)
# then set it right away.
# $ variable refs are set after all checks have run during the
# generating step.
if [[ ${#5} != 0 ]] ; then
if [[ $5 = *\$* ]]; then
mkl_var_append "MKL_LATE_VARS" "opt_$safeopt:$5"
else
opt_$safeopt $5
fi
fi
if [[ ! -z $varname ]]; then
# Add variable to list
MKL_CONFVARS="$MKL_CONFVARS $varname"
fi
}
# Adds a toggle (--enable-X, --disable-X) option.
# Arguments:
# option group ("Standard", ..)
# variable name (WITH_FOO)
# option (--enable-foo)
# help ("foo.." ("Enable" and "Disable" will be prepended))
# default (y or n)
function mkl_toggle_option {
# Add option argument
mkl_option "$1" "$2" "$3" "$4" "$5"
# Add corresponding "--disable-foo" option for "--enable-foo".
local disname="${3/--enable/--disable}"
local dishelp="${4/Enable/Disable}"
mkl_option "$1" "$2" "$disname" "$dishelp" "" "n"
}
# Adds a toggle (--enable-X, --disable-X) option with builtin checker.
# This is the library version.
# Arguments:
# option group ("Standard", ..)
# config name (foo, must be same as pkg-config name)
# variable name (WITH_FOO)
# action (fail or disable)
# option (--enable-foo)
# help (defaults to "Enable ")
# linker flags (-lfoo)
# default (y or n)
function mkl_toggle_option_lib {
local help="$6"
[[ -z "$help" ]] && help="Enable $2"
# Add option argument
mkl_option "$1" "$3" "$5" "$help" "$8"
# Add corresponding "--disable-foo" option for "--enable-foo".
local disname="${5/--enable/--disable}"
local dishelp="${help/Enable/Disable}"
mkl_option "$1" "$3" "$disname" "$dishelp" "" "n"
# Create checks
eval "function _tmp_func { mkl_lib_check \"$2\" \"$3\" \"$4\" CC \"$7\"; }"
mkl_func_push MKL_CHECKS "$MKL_MODULE" _tmp_func
}
librdkafka-0.9.3/mklove/modules/configure.builtin 0000664 0000000 0000000 00000005631 13041406524 0022137 0 ustar 00root root 0000000 0000000 #!/bin/bash
#
# mklove builtin checks and options
# Sets:
# prefix, etc..
mkl_option "Standard" prefix "--prefix=PATH" \
"Install arch-independent files in PATH" "/usr/local"
mkl_option "Standard" exec_prefix "--exec-prefix=PATH" \
"Install arch-dependent files in PATH" "\$prefix"
mkl_option "Standard" bindir "--bindir=PATH" "User executables" "\$exec_prefix/bin"
mkl_option "Standard" sbindir "--sbindir=PATH" "System admin executables" \
"\$exec_prefix/sbin"
mkl_option "Standard" libexecdir "--libexecdir=PATH" "Program executables" \
"\$exec_prefix/libexec"
mkl_option "Standard" datadir "--datadir=PATH" "Read-only arch-independent data" \
"\$prefix/share"
mkl_option "Standard" sysconfdir "--sysconfdir=PATH" "Configuration data" \
"\$prefix/etc"
mkl_option "Standard" sharedstatedir "--sharedstatedir=PATH" \
"Modifiable arch-independent data" "\$prefix/com"
mkl_option "Standard" localstatedir "--localstatedir=PATH" \
"Modifiable local state data" "\$prefix/var"
mkl_option "Standard" libdir "--libdir=PATH" "Libraries" "\$exec_prefix/lib"
mkl_option "Standard" includedir "--includedir=PATH" "C/C++ header files" \
"\$prefix/include"
mkl_option "Standard" infodir "--infodir=PATH" "Info documentation" "\$prefix/info"
mkl_option "Standard" mandir "--mandir=PATH" "Manual pages" "\$prefix/man"
mkl_option "Configure tool" "" "--list-modules" "List loaded mklove modules"
mkl_option "Configure tool" "" "--list-checks" "List checks"
mkl_option "Configure tool" env:MKL_FAILFATAL "--fail-fatal" "All failures are fatal"
mkl_option "Configure tool" env:MKL_NOCACHE "--no-cache" "Dont use or generate config.cache"
mkl_option "Configure tool" env:MKL_DEBUG "--debug" "Enable configure debugging"
mkl_option "Configure tool" env:MKL_CLEAN "--clean" "Remove generated configure files"
mkl_option "Configure tool" "" "--reconfigure" "Rerun configure with same arguments as last run"
mkl_option "Configure tool" env:MKL_NO_DOWNLOAD "--no-download" "Disable downloads of required mklove modules"
mkl_option "Configure tool" env:MKL_UPDATE_MODS "--update-modules" "Update modules from global repository"
mkl_option "Configure tool" env:MKL_REPO_URL "--repo-url=URL_OR_PATH" "Override mklove modules repo URL" "$MKL_REPO_URL"
mkl_option "Configure tool" "" "--help" "Show configure usage"
mkl_toggle_option "Compatibility" "mk:MKL_MAINT_MODE" "--enable-maintainer-mode" "Maintainer mode (no-op)"
mkl_option "Configure tool" "mk:PROGRAM_PREFIX" "--program-prefix=PFX" "Program prefix"
mkl_option "Compatibility" "mk:DISABL_DEP_TRACK" "--disable-dependency-tracking" "Disable dependency tracking (no-op)"
mkl_option "Compatibility" "mk:DISABL_SILENT_RULES" "--disable-silent-rules" "Verbose build output (no-op)"
function checks {
if [[ ! -z $libdir ]]; then
mkl_mkvar_append "libdir" LDFLAGS "-L${libdir}"
fi
if [[ ! -z $includedir ]]; then
mkl_mkvar_append "includedir" CPPFLAGS "-I${includedir}"
fi
}
librdkafka-0.9.3/mklove/modules/configure.cc 0000664 0000000 0000000 00000013716 13041406524 0021061 0 ustar 00root root 0000000 0000000 #!/bin/bash
#
# Compiler detection
# Sets:
# CC, CXX, CFLAGS, CPPFLAGS, LDFLAGS, ARFLAGS, PKG_CONFIG, INSTALL, MBITS
mkl_require host
function checks {
# C compiler
mkl_meta_set "ccenv" "name" "C compiler from CC env"
if ! mkl_command_check "ccenv" "WITH_CC" cont "$CC --version"; then
if mkl_command_check "gcc" "WITH_GCC" cont "gcc --version"; then
CC=gcc
elif mkl_command_check "clang" "WITH_CLANG" cont "clang --version"; then
CC=clang
elif mkl_command_check "cc" "WITH_CC" fail "cc --version"; then
CC=cc
fi
fi
export CC="${CC}"
mkl_mkvar_set CC CC "$CC"
if [[ $MKL_CC_WANT_CXX == 1 ]]; then
# C++ compiler
mkl_meta_set "cxxenv" "name" "C++ compiler from CXX env"
if ! mkl_command_check "cxxenv" "WITH_CXX" cont "$CXX --version" ; then
mkl_meta_set "gxx" "name" "C++ compiler (g++)"
mkl_meta_set "clangxx" "name" "C++ compiler (clang++)"
mkl_meta_set "cxx" "name" "C++ compiler (c++)"
if mkl_command_check "gxx" "WITH_GXX" cont "g++ --version"; then
CXX=g++
elif mkl_command_check "clangxx" "WITH_CLANGXX" cont "clang++ --version"; then
CXX=clang++
elif mkl_command_check "cxx" "WITH_CXX" fail "c++ --version"; then
CXX=c++
fi
fi
export CXX="${CXX}"
mkl_mkvar_set "CXX" CXX $CXX
fi
# Handle machine bits, if specified.
if [[ ! -z "$MBITS" ]]; then
mkl_meta_set mbits_m name "mbits compiler flag (-m$MBITS)"
if mkl_compile_check mbits_m "" fail CC "-m$MBITS"; then
mkl_mkvar_append CPPFLAGS CPPFLAGS "-m$MBITS"
mkl_mkvar_append LDFLAGS LDFLAGS "-m$MBITS"
fi
if [[ -z "$ARFLAGS" && $MBITS == 64 && $MKL_DISTRO == "SunOS" ]]; then
# Turn on 64-bit archives on SunOS
mkl_mkvar_append ARFLAGS ARFLAGS "S"
fi
fi
# Provide prefix and checks for various other build tools.
local t=
for t in LD:ld NM:nm OBJDUMP:objdump STRIP:strip ; do
local tenv=${t%:*}
t=${t#*:}
local tval="${!tenv}"
[[ -z $tval ]] && tval="$t"
if mkl_prog_check "$t" "" disable "$tval" ; then
if [[ $tval != ${!tenv} ]]; then
export "$tenv"="$tval"
fi
mkl_mkvar_set $tenv $tenv "$tval"
fi
done
# Compiler and linker flags
[[ ! -z $CFLAGS ]] && mkl_mkvar_set "CFLAGS" "CFLAGS" "$CFLAGS"
[[ ! -z $CPPFLAGS ]] && mkl_mkvar_set "CPPFLAGS" "CPPFLAGS" "$CPPFLAGS"
[[ ! -z $CXXFLAGS ]] && mkl_mkvar_set "CXXFLAGS" "CXXFLAGS" "$CXXFLAGS"
[[ ! -z $LDFLAGS ]] && mkl_mkvar_set "LDFLAGS" "LDFLAGS" "$LDFLAGS"
[[ ! -z $ARFLAGS ]] && mkl_mkvar_set "ARFLAGS" "ARFLAGS" "$ARFLAGS"
if [[ $MKL_NO_DEBUG_SYMBOLS != "y" ]]; then
# Add debug symbol flag (-g)
# OSX 10.9 requires -gstrict-dwarf for some reason.
mkl_meta_set cc_g_dwarf name "debug symbols compiler flag (-g...)"
if [[ $MKL_DISTRO == "osx" ]]; then
if mkl_compile_check cc_g_dwarf "" cont CC "-gstrict-dwarf"; then
mkl_mkvar_append CPPFLAGS CPPFLAGS "-gstrict-dwarf"
else
mkl_mkvar_append CPPFLAGS CPPFLAGS "-g"
fi
else
mkl_mkvar_append CPPFLAGS CPPFLAGS "-g"
fi
fi
# pkg-config
if [ -z "$PKG_CONFIG" ]; then
PKG_CONFIG=pkg-config
fi
if mkl_command_check "pkgconfig" "WITH_PKGCONFIG" cont "$PKG_CONFIG --version"; then
export PKG_CONFIG
fi
mkl_mkvar_set "pkgconfig" PKG_CONFIG $PKG_CONFIG
[[ ! -z "$PKG_CONFIG_PATH" ]] && mkl_env_append PKG_CONFIG_PATH "$PKG_CONFIG_PATH"
# install
if [ -z "$INSTALL" ]; then
if [[ $MKL_DISTRO == "SunOS" ]]; then
mkl_meta_set ginstall name "GNU install"
if mkl_command_check ginstall "" ignore "ginstall --version"; then
INSTALL=ginstall
else
INSTALL=install
fi
else
INSTALL=install
fi
fi
if mkl_command_check "install" "WITH_INSTALL" cont "$INSTALL --version"; then
export INSTALL
fi
mkl_mkvar_set "install" INSTALL $INSTALL
# Enable profiling if desired
if [[ $WITH_PROFILING == y ]]; then
mkl_allvar_set "" "WITH_PROFILING" "y"
mkl_mkvar_append CPPFLAGS CPPFLAGS "-pg"
mkl_mkvar_append LDFLAGS LDFLAGS "-pg"
fi
# Optimization
if [[ $WITHOUT_OPTIMIZATION == n ]]; then
mkl_mkvar_append CPPFLAGS CPPFLAGS "-O2"
else
mkl_mkvar_append CPPFLAGS CPPFLAGS "-O0"
fi
# Static linking
if [[ $WITH_STATIC_LINKING == y ]]; then
# LDFLAGS_STATIC is the LDFLAGS needed to enable static linking
# of sub-sequent libraries, while
# LDFLAGS_DYNAMIC is the LDFLAGS needed to enable dynamic linking.
mkl_mkvar_set staticlinking LDFLAGS_STATIC "-Wl,-Bstatic"
mkl_mkvar_set staticlinking LDFLAGS_DYNAMIC "-Wl,-Bdynamic"
fi
}
mkl_option "Compiler" "env:CC" "--cc=CC" "Build using C compiler CC" "\$CC"
mkl_option "Compiler" "env:CXX" "--cxx=CXX" "Build using C++ compiler CXX" "\$CXX"
mkl_option "Compiler" "ARCH" "--arch=ARCH" "Build for architecture" "$(uname -m)"
mkl_option "Compiler" "CPU" "--cpu=CPU" "Build and optimize for specific CPU" "generic"
mkl_option "Compiler" "MBITS" "--mbits=BITS" "Machine bits (32 or 64)" ""
for n in CFLAGS CPPFLAGS CXXFLAGS LDFLAGS ARFLAGS; do
mkl_option "Compiler" "mk:$n" "--$n=$n" "Add $n flags"
done
mkl_option "Compiler" "env:PKG_CONFIG_PATH" "--pkg-config-path" "Extra paths for pkg-config"
mkl_option "Compiler" "WITH_PROFILING" "--enable-profiling" "Enable profiling"
mkl_option "Compiler" "WITH_STATIC_LINKING" "--enable-static" "Enable static linking"
mkl_option "Compiler" "WITHOUT_OPTIMIZATION" "--disable-optimization" "Disable optimization flag to compiler" "n"
mkl_option "Compiler" "env:MKL_NO_DEBUG_SYMBOLS" "--disable-debug-symbols" "Disable debugging symbols" "n"
mkl_option "Compiler" "env:MKL_WANT_WERROR" "--enable-werror" "Enable compiler warnings as errors" "n"
librdkafka-0.9.3/mklove/modules/configure.cxx 0000664 0000000 0000000 00000000153 13041406524 0021265 0 ustar 00root root 0000000 0000000 #!/bin/bash
#
# C++ detection
#
# This script simply limits the checks of configure.cc
MKL_CC_WANT_CXX=1
librdkafka-0.9.3/mklove/modules/configure.fileversion 0000664 0000000 0000000 00000004021 13041406524 0023006 0 ustar 00root root 0000000 0000000 #!/bin/bash
#
# Reads version from file and sets variables accordingly
# The first non-commented line in the file is expected to be the version string.
# Arguments:
# filename
# STR_VERSION_VARIABLE_NAME
# [ HEX_VERSION_VARIABLE_NAME ]
#
# Example: Set string version in variable named "MYVERSION_STR" and
# the hex representation in "MYVERSION"
# mkl_require VERSION.txt MYVERSION_STR MYVERSION
if [[ -z "$2" ]]; then
mkl_fail "fileversion" "none" "fail" "Missing argument(s), expected: FILENAME STR_VER HEX_VER"
return 0
fi
fileversion_file="$1"
fileversion_strvar="$2"
fileversion_hexvar="$3"
function checks {
mkl_check_begin "fileversion" "" "no-cache" "version from file $fileversion_file"
if [[ ! -s $fileversion_file ]]; then
mkl_check_failed "fileversion" "" "fail" \
"Version file $fileversion_file is not readable"
return 1
fi
local orig=$(grep -v ^\# "$fileversion_file" | grep -v '^$' | head -1)
# Strip v prefix if any
orig=${orig#v}
# Try to decode version string into hex
# Supported format is "[v]NN.NN.NN[.NN]"
if [[ ! -z $fileversion_hexvar ]]; then
local hex=""
local s=${orig#v} # Strip v prefix, if any.
local ncnt=0
local n=
for n in ${s//./ } ; do
if [[ ! ( "$n" =~ ^[0-9][0-9]?$ ) ]]; then
mkl_check_failed "fileversion" "" "fail" \
"$fileversion_file: Could not decode '$orig' into hex version, expecting format 'NN.NN.NN[.NN]'"
return 1
fi
hex="$hex$(printf %02x $n)"
ncnt=$(expr $ncnt + 1)
done
if [[ ! -z $hex ]]; then
# Finish all four bytess
for n in {$ncnt..4} ; do
hex="$hex$(printf %02x 0)"
done
mkl_allvar_set "fileversion" "$fileversion_hexvar" "0x$hex"
fi
fi
mkl_allvar_set "fileversion" "$fileversion_strvar" "$orig"
mkl_check_done "fileversion" "" "cont" "ok" "${!fileversion_strvar}"
}
librdkafka-0.9.3/mklove/modules/configure.gitversion 0000664 0000000 0000000 00000000674 13041406524 0022664 0 ustar 00root root 0000000 0000000 #!/bin/bash
#
# Sets version variable from git information.
# Optional arguments:
# "as"
# VARIABLE_NAME
#
# Example: Set version in variable named "MYVERSION":
# mkl_require gitversion as MYVERSION
if [[ $1 == "as" ]]; then
__MKL_GITVERSION_VARNAME="$2"
else
__MKL_GITVERSION_VARNAME="VERSION"
fi
function checks {
mkl_allvar_set "gitversion" "$__MKL_GITVERSION_VARNAME" "$(git describe --abbrev=6 --tags HEAD --always)"
}
librdkafka-0.9.3/mklove/modules/configure.good_cflags 0000664 0000000 0000000 00000000513 13041406524 0022732 0 ustar 00root root 0000000 0000000 #!/bin/bash
#
# Provides some known-good CFLAGS
# Sets:
# CFLAGS
# CXXFLAGS
# CPPFLAGS
function checks {
mkl_mkvar_append CPPFLAGS CPPFLAGS \
"-Wall -Wsign-compare -Wfloat-equal -Wpointer-arith"
if [[ $MKL_WANT_WERROR = "y" ]]; then
mkl_mkvar_append CPPFLAGS CPPFLAGS \
"-Werror"
fi
}
librdkafka-0.9.3/mklove/modules/configure.host 0000664 0000000 0000000 00000005366 13041406524 0021453 0 ustar 00root root 0000000 0000000 #!/bin/bash
#
# Host OS support
# Sets:
# HOST
# BUILD
# TARGET
# FIXME: No need for this right now
#mkl_require host_linux
#mkl_require host_osx
#mkl_require host_cygwin
#mkl_option "Cross-compilation" "mk:HOST_OS" "--host-os=osname" "Host OS (linux,osx,cygwin,..)" "auto"
# autoconf compatibility - does nothing at this point
mkl_option "Cross-compilation" "mk:HOST" "--host=HOST" "Configure to build programs to run on HOST (no-op)"
mkl_option "Cross-compilation" "mk:BUILD" "--build=BUILD" "Configure for building on BUILD (no-op)"
mkl_option "Cross-compilation" "mk:TARGET" "--target=TARGET" "Configure for building cross-toolkits for platform TARGET (no-op)"
function checks {
# Try to figure out what OS/distro we are running on.
mkl_check_begin "distro" "" "no-cache" "OS or distribution"
# Try lsb_release
local sys
sys=$(lsb_release -is 2>/dev/null)
if [[ $? -gt 0 ]]; then
# That didnt work, try uname.
local kn=$(uname -s)
case $kn in
Linux)
sys=Linux
;;
Darwin)
sys=osx
;;
CYGWIN*)
sys=Cygwin
;;
*)
sys="$kn"
;;
esac
fi
if [[ -z $sys ]]; then
mkl_check_failed "distro" "" "ignore" ""
else
mkl_check_done "distro" "" "ignore" "ok" "$sys"
mkl_mkvar_set "distro" "MKL_DISTRO" "$sys"
fi
}
#function checks {
# mkl_check_begin "host" "HOST_OS" "no-cache" "host OS"
#
# #
# # If --host-os=.. was not specified then this is most likely not a
# # a cross-compilation and we can base the host-os on the native OS.
# #
# if [[ $HOST_OS != "auto" ]]; then
# mkl_check_done "host" "HOST_OS" "cont" "ok" "$HOST_OS"
# return 0
# fi
#
# kn=$(uname -s)
# case $kn in
# Linux)
# hostos=linux
# ;;
# Darwin)
# hostos=osx
# ;;
# CYGWIN*)
# hostos=cygwin
# ;;
# *)
# hostos="$(mkl_lower $kn)"
# mkl_err "Unknown host OS kernel name: $kn"
# mkl_err0 " Will attempt to load module host_$hostos anyway."
# mkl_err0 " Please consider writing a configure.host_$hostos"
# ;;
# esac
#
# if ! mkl_require --try "host_$hostos"; then
# # Module not found
# mkl_check_done "host" "HOST_OS" "cont" "failed" "$kn?"
# else
# # Module loaded
#
# if mkl_func_exists "host_${hostos}_setup" ; then
# "host_${hostos}_setup"
# fi
#
# mkl_check_done "host" "HOST_OS" "cont" "ok" "$hostos"
# fi
#
# # Set HOST_OS var even if probing failed.
# mkl_mkvar_set "host" "HOST_OS" "$hostos"
#}
librdkafka-0.9.3/mklove/modules/configure.lib 0000664 0000000 0000000 00000002735 13041406524 0021241 0 ustar 00root root 0000000 0000000 #!/bin/bash
#
# Module for building shared libraries
# Sets:
# WITH_GNULD | WITH_OSXLD
# WITH_LDS - linker script support
mkl_require pic
function checks {
mkl_mkvar_append LIB_LDFLAGS LIB_LDFLAGS '-shared'
# Check what arguments to pass to CC or LD for shared libraries
mkl_meta_set gnulib name "GNU-compatible linker options"
mkl_meta_set osxlib name "OSX linker options"
if mkl_compile_check gnulib WITH_GNULD cont CC \
"-shared -Wl,-soname,mkltest.0" "" ; then
# GNU linker
mkl_mkvar_append LIB_LDFLAGS LIB_LDFLAGS '-Wl,-soname,$(LIBFILENAME)'
elif mkl_compile_check osxlib WITH_OSXLD cont CC \
"-dynamiclib -Wl,-install_name,/tmp/mkltest.so.0" ; then
# OSX linker
mkl_mkvar_append LIB_LDFLAGS LIB_LDFLAGS '-dynamiclib -Wl,-install_name,$(DESTDIR)$(libdir)/$(LIBFILENAME)'
fi
# Check what argument is needed for passing linker script.
local ldsfile=$(mktemp _mkltmpXXXXXX)
echo "{
global:
*;
};
" > $ldsfile
mkl_meta_set ldsflagvs name "GNU linker-script ld flag"
mkl_meta_set ldsflagm name "Solaris linker-script ld flag"
if mkl_compile_check ldsflagvs "" cont CC \
"-shared -Wl,--version-script=$ldsfile"; then
mkl_mkvar_set ldsflagvs LDFLAG_LINKERSCRIPT "-Wl,--version-script="
mkl_mkvar_set lib_lds WITH_LDS y
elif mkl_compile_check ldsflagm "" ignore CC \
"-shared -Wl,-M$ldsfile"; then
mkl_mkvar_set ldsflagm LDFLAG_LINKERSCRIPT "-Wl,-M"
mkl_mkvar_set lib_lds WITH_LDS y
fi
rm -f "$ldsfile"
}
librdkafka-0.9.3/mklove/modules/configure.parseversion 0000664 0000000 0000000 00000005271 13041406524 0023211 0 ustar 00root root 0000000 0000000 #!/bin/bash
#
# Parses the provided version string and creates variables accordingly.
# [ "hex2str" ] -- version-string is in hex (e.g., 0x00080300)
# version-string
# STR_VERSION_VARIABLE_NAME
# [ HEX_VERSION_VARIABLE_NAME ]
#
# Note: The version will also be set in MKL_APP_VERSION
#
# Example: Set string version in variable named "MYVERSION_STR" and
# the hex representation in "MYVERSION"
# mkl_require parseversion "$(head -1 VERSION.txt)" MYVERSION_STR MYVERSION
if [[ $1 == "hex2str" ]]; then
parseversion_type="hex"
parseversion_fmt="${2}:END:%d%d%d%d"
shift
shift
else
parseversion_type=""
parseversion_fmt="%d.%d.%d.%d"
fi
if [[ -z "$2" ]]; then
mkl_fail "parseversion" "none" "fail" "Missing argument(s)"
return 0
fi
parseversion_orig="$1"
parseversion_strvar="$2"
parseversion_hexvar="$3"
function checks {
mkl_check_begin --verb "parsing" "parseversion" "" "no-cache" \
"version '$parseversion_orig'"
# Strip v prefix if any
orig=${parseversion_orig#v}
if [[ $orig == 0x* ]]; then
parseversion_type="hex"
orig=${orig#0x}
fi
if [[ -z $orig ]]; then
mkl_check_failed "parseversion" "" "fail" "Version string is empty"
return 1
fi
# If orig is in hex we construct a string format instead.
if [[ $parseversion_type == "hex" ]]; then
local s=$orig
local str=""
local vals=""
while [[ ! -z $s ]]; do
local n=${s:0:2}
s=${s:${#n}}
vals="${vals}$(printf %d 0x$n) "
done
str=$(printf "$parseversion_fmt" $vals)
orig=${str%:END:*}
fi
# Try to decode version string into hex
# Supported format is "[v]NN.NN.NN[.NN]"
if [[ ! -z $parseversion_hexvar ]]; then
local hex=""
local s=$orig
local ncnt=0
local n=
for n in ${s//./ } ; do
if [[ ! ( "$n" =~ ^[0-9][0-9]?$ ) ]]; then
mkl_check_failed "parseversion" "" "fail" \
"Could not decode '$parseversion_orig' into hex version, expecting format 'NN.NN.NN[.NN]'"
return 1
fi
hex="$hex$(printf %02x $n)"
ncnt=$(expr $ncnt + 1)
done
if [[ ! -z $hex ]]; then
# Finish all four bytess
while [[ ${#hex} -lt 8 ]]; do
hex="$hex$(printf %02x 0)"
done
mkl_allvar_set "parseversion" "$parseversion_hexvar" "0x$hex"
fi
fi
mkl_allvar_set "parseversion" "$parseversion_strvar" "$orig"
mkl_allvar_set "parseversion" MKL_APP_VERSION "$orig"
mkl_check_done "parseversion" "" "cont" "ok" "${!parseversion_strvar}"
}
librdkafka-0.9.3/mklove/modules/configure.pic 0000664 0000000 0000000 00000000375 13041406524 0021244 0 ustar 00root root 0000000 0000000 #!/bin/bash
#
# Checks if -fPIC is supported, and if so turns it on.
#
# Sets:
# HAVE_PIC
# CPPFLAGS
#
function checks {
if mkl_compile_check PIC HAVE_PIC disable CC "-fPIC" "" ; then
mkl_mkvar_append CPPFLAGS CPPFLAGS "-fPIC"
fi
}
librdkafka-0.9.3/mklove/modules/configure.socket 0000664 0000000 0000000 00000000720 13041406524 0021753 0 ustar 00root root 0000000 0000000 #!/bin/bash
#
# Provides proper compiler flags for socket support, e.g. socket(3).
function checks {
local src="
#include
#include
#include
void foo (void) {
int s = socket(0, 0, 0);
close(s);
}"
if ! mkl_compile_check socket "" cont CC "" "$src"; then
if mkl_compile_check --ldflags="-lsocket -lnsl" socket_nsl "" fail CC "" "$src"; then
mkl_mkvar_append socket_nsl LIBS "-lsocket -lnsl"
fi
fi
}
librdkafka-0.9.3/packaging/ 0000775 0000000 0000000 00000000000 13041406524 0015540 5 ustar 00root root 0000000 0000000 librdkafka-0.9.3/packaging/archlinux/ 0000775 0000000 0000000 00000000000 13041406524 0017535 5 ustar 00root root 0000000 0000000 librdkafka-0.9.3/packaging/archlinux/PKGBUILD 0000664 0000000 0000000 00000000157 13041406524 0020664 0 ustar 00root root 0000000 0000000 pkgname=librdkafka
pkgver=master
pkgrel=1
pkgdesc=The Apache Kafka C/C++ client library
arch=('i686' 'x86_64')
librdkafka-0.9.3/packaging/debian/ 0000775 0000000 0000000 00000000000 13041406524 0016762 5 ustar 00root root 0000000 0000000 librdkafka-0.9.3/packaging/debian/.gitignore 0000664 0000000 0000000 00000000073 13041406524 0020752 0 ustar 00root root 0000000 0000000 *.log
files
librdkafka-dev
librdkafka1-dbg
librdkafka1
tmp
librdkafka-0.9.3/packaging/debian/changelog 0000664 0000000 0000000 00000004774 13041406524 0020650 0 ustar 00root root 0000000 0000000 librdkafka (0.8.6-1) unstable; urgency=medium
* New upstream release.
* Backport upstream commit f6fd0da, adding --disable-silent-rules
compatibility support to mklove. (Closes: #788742)
-- Faidon Liambotis Sun, 19 Jul 2015 01:36:18 +0300
librdkafka (0.8.5-2) unstable; urgency=medium
* Install rdkafka.pc in the right, multiarch location. (Closes: #766759)
-- Faidon Liambotis Sun, 26 Oct 2014 06:47:07 +0200
librdkafka (0.8.5-1) unstable; urgency=medium
* New upstream release.
- Fixes kFreeBSD FTBFS.
* Ship rdkafka.pc pkg-config in librdkafka-dev.
-- Faidon Liambotis Fri, 24 Oct 2014 18:03:22 +0300
librdkafka (0.8.4-1) unstable; urgency=medium
* New upstream release, including a new build system.
- Add Build-Depends on perl, required by configure.
- Support multiarch library paths.
- Better detection of architecture atomic builtins, supporting more
architectures. (Closes: #739930)
- Various portability bugs fixed. (Closes: #730506)
- Update debian/librdkafka1.symbols.
* Convert to a multiarch package.
* Switch to Architecture: any, because of renewed upstream portability.
* Update debian/copyright to add src/ before Files: paths.
* Update Standards-Version to 3.9.6, no changes needed.
* Ship only the C library for now, not the new C++ library; the latter is
still in flux in some ways and will probably be shipped in a separate
package in a future release.
-- Faidon Liambotis Wed, 22 Oct 2014 23:57:24 +0300
librdkafka (0.8.3-1) unstable; urgency=medium
* New upstream release.
- Multiple internal symbols hidden; breaks ABI without a SONAME bump, but
these were internal and should not break any applications, packaged or
not.
* Update Standards-Version to 3.9.5, no changes needed.
-- Faidon Liambotis Tue, 18 Feb 2014 02:21:43 +0200
librdkafka (0.8.1-1) unstable; urgency=medium
* New upstream release.
- Multiple fixes to FTBFS on various architectures. (Closes: #730506)
- Remove dh_auto_clean override, fixed upstream.
* Limit the set of architectures: upstream currently relies on 64-bit atomic
operations that several Debian architectures do not support.
-- Faidon Liambotis Thu, 05 Dec 2013 16:53:28 +0200
librdkafka (0.8.0-1) unstable; urgency=low
* Initial release. (Closes: #710271)
-- Faidon Liambotis Mon, 04 Nov 2013 16:50:07 +0200
librdkafka-0.9.3/packaging/debian/compat 0000664 0000000 0000000 00000000002 13041406524 0020160 0 ustar 00root root 0000000 0000000 9
librdkafka-0.9.3/packaging/debian/control 0000664 0000000 0000000 00000003520 13041406524 0020365 0 ustar 00root root 0000000 0000000 Source: librdkafka
Priority: optional
Maintainer: Faidon Liambotis
Build-Depends: debhelper (>= 9), zlib1g-dev, libssl-dev, libsasl2-dev, python
Standards-Version: 3.9.6
Section: libs
Homepage: https://github.com/edenhill/librdkafka
Vcs-Git: git://github.com/edenhill/librdkafka.git -b debian
Vcs-Browser: https://github.com/edenhill/librdkafka/tree/debian
Package: librdkafka1
Architecture: any
Multi-Arch: same
Depends: ${shlibs:Depends}, ${misc:Depends}
Description: library implementing the Apache Kafka protocol
librdkafka is a C implementation of the Apache Kafka protocol. It currently
implements the 0.8 version of the protocol and can be used to develop both
Producers and Consumers.
.
More information about Apache Kafka can be found at http://kafka.apache.org/
Package: librdkafka-dev
Section: libdevel
Architecture: any
Multi-Arch: same
Depends: librdkafka1 (= ${binary:Version}), ${misc:Depends}
Description: library implementing the Apache Kafka protocol (development headers)
librdkafka is a C implementation of the Apache Kafka protocol. It currently
implements the 0.8 version of the protocol and can be used to develop both
Producers and Consumers.
.
More information about Apache Kafka can be found at http://kafka.apache.org/
.
This package contains the development headers.
Package: librdkafka1-dbg
Section: debug
Priority: extra
Architecture: any
Multi-Arch: same
Depends: librdkafka1 (= ${binary:Version}), ${misc:Depends}
Description: library implementing the Apache Kafka protocol (debugging symbols)
librdkafka is a C implementation of the Apache Kafka protocol. It currently
implements the 0.8 version of the protocol and can be used to develop both
Producers and Consumers.
.
More information about Apache Kafka can be found at http://kafka.apache.org/
.
This package contains the debugging symbols.
librdkafka-0.9.3/packaging/debian/copyright 0000664 0000000 0000000 00000010457 13041406524 0020724 0 ustar 00root root 0000000 0000000 Format: http://www.debian.org/doc/packaging-manuals/copyright-format/1.0/
Upstream-Name: librdkafka
Source: https://github.com/edenhill/librdkafka
License: BSD-2-clause
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
.
1. Redistributions of source code must retain the above copyright notice,
this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
POSSIBILITY OF SUCH DAMAGE.
Files: *
Copyright: 2012-2015, Magnus Edenhill
License: BSD-2-clause
Files: src/rdcrc32.c src/rdcrc32.h
Copyright: 2006-2012, Thomas Pircher
License: MIT
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
.
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
Files: src/snappy.c src/snappy.h src/snappy_compat.h
Copyright: 2005, Google Inc.
2011, Intel Corporation
License: BSD-3-clause
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
met:
.
* Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above
copyright notice, this list of conditions and the following disclaimer
in the documentation and/or other materials provided with the
distribution.
* Neither the name of Google Inc. nor the names of its
contributors may be used to endorse or promote products derived from
this software without specific prior written permission.
.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Files: debian/*
Copyright: 2013 Faidon Liambotis
License: BSD-2-clause
librdkafka-0.9.3/packaging/debian/docs 0000664 0000000 0000000 00000000053 13041406524 0017633 0 ustar 00root root 0000000 0000000 README.md
INTRODUCTION.md
CONFIGURATION.md
librdkafka-0.9.3/packaging/debian/gbp.conf 0000664 0000000 0000000 00000000317 13041406524 0020402 0 ustar 00root root 0000000 0000000 [buildpackage]
upstream-tree=tag
upstream-branch=master
debian-branch=debian
upstream-tag=%(version)s
debian-tag=debian/%(version)s
no-create-orig = True
tarball-dir = ../tarballs
export-dir = ../build-area
librdkafka-0.9.3/packaging/debian/librdkafka-dev.dirs 0000664 0000000 0000000 00000000024 13041406524 0022507 0 ustar 00root root 0000000 0000000 usr/lib
usr/include
librdkafka-0.9.3/packaging/debian/librdkafka-dev.examples 0000664 0000000 0000000 00000000072 13041406524 0023367 0 ustar 00root root 0000000 0000000 examples/rdkafka_example.c
examples/rdkafka_performance.c
librdkafka-0.9.3/packaging/debian/librdkafka-dev.install 0000664 0000000 0000000 00000000225 13041406524 0023217 0 ustar 00root root 0000000 0000000 usr/include/*/rdkafka.h
usr/include/*/rdkafkacpp.h
usr/lib/*/librdkafka.a
usr/lib/*/librdkafka.so
usr/lib/*/librdkafka++.a
usr/lib/*/librdkafka++.so
librdkafka-0.9.3/packaging/debian/librdkafka-dev.substvars 0000664 0000000 0000000 00000000016 13041406524 0023603 0 ustar 00root root 0000000 0000000 misc:Depends=
librdkafka-0.9.3/packaging/debian/librdkafka.dsc 0000664 0000000 0000000 00000001227 13041406524 0021551 0 ustar 00root root 0000000 0000000 Format: 3.0 (quilt)
Source: librdkafka
Binary: librdkafka1, librdkafka-dev, librdkafka1-dbg
Architecture: any
Version: 0.9.1-1pre1
Maintainer: Magnus Edenhill