• Breaking News

    Sunday, July 5, 2020

    Which is better, CSV or JSON for migrating large data to another database? Ask Programming

    Which is better, CSV or JSON for migrating large data to another database? Ask Programming


    Which is better, CSV or JSON for migrating large data to another database?

    Posted: 05 Jul 2020 07:17 AM PDT

    I need to import about 2 million records into an existing SQL Server database table that already has records in it. The new records to import are coming from a third-party company and I need to specify if I want this information in a JSON or CSV or XML file. I am unsure which to choose.

    My research thus far has me leaning toward CSV. This is because apparently, with JSON, the JSON object would have to be loaded entirely into memory first before being able to perform transform or insert operations. With two million records having 20 fields each, this seems like it could cause memory issues. On the other hand, with CSV, apparently each record can be read one at a time, which seems must more manageable.

    For some added context, the data to import is non-hierarchical, and uses strings, dates, numbers and booleans. The strings aren't longer than 250 characters, the dates are datetimes (with UTC offsets), and the numbers range only from 0 to 100,000 (inclusive).

    Do you agree with my assumptions about memory usage? Can anyone here with experience doing these operations think of anything else I should specify that I need in terms of formatting the data?

    submitted by /u/dlebauche
    [link] [comments]

    Please advise me on the best way to architect this simple application.

    Posted: 05 Jul 2020 05:44 PM PDT

    I am currently building a habit tracking application to help me learn Node. Before I get too far along with the application I am hoping to get some advice on how to build it out.

    The app is being built on the MERN stack, so the db will be collection/JSON based.

    The application allows users to create daily habits, and give them a numerical value, and decide if the value should be positive or negative. So each day there is a counter that is affected by the habit. One could choose to make smoking cigarettes worth -10 points, and make meditation worth +10, and so forth.

    I'm unsure of the best way to setup the database.

    Currently I have two collections.

    My user collection which looks like:

    users: [ id { name: name, email: email, password: password } ] 

    and habits:

     [ id { name: name, description: description, points: points habitType: habitType user: (references the user collection to associate a habit with a user) } ] 

    As I move forward I need the application to store whether or not a habit was completed for the day, and I'm wondering if I should do that by adding a date object to the user array - and then each time a habit is completed, adding to that object. That way the front end can query this for the daily counter.

    submitted by /u/jaypeejay
    [link] [comments]

    Designing a proper concurrent system with sync (Java)

    Posted: 05 Jul 2020 09:21 PM PDT

    Hello, all!

    First time posting here so I hope the format of my question is correct, if I made a mistake please let me know.

    I am writing a program which works as follows:

    There exist message objects which some threads I call subscribers must handle and publisher threads create.

    Message objects must be acted on in the order that they are pushed to each thread's input queue.

    Some of these message objects hold a sync flag and a list of threads which all receive this identical message and must all work on this object at the exact same time. Note that there are times when multiple threads receive the same message without sync, and at any time any thread can have a backlog of multiple sync and non-sync messages.

    That is, when a thread receives a sync message, the members of the sync group defined in the message must wait until all members have finished with the rest of their messages from before the sync message in order to act on the sync message at the same time.

    My solution to this has been very janky. What I did is create a hash table with the key being the message object and the values an arraylist of threads. Each thread adds itself to the hashtable's arraylist when it gets the sync grouped message and all threads poll the hashtable's size until the correct number of threads is present, and then they work on the message. That is, when a thread is added to the ArrayList, it polls the size. If it is not adequate, it awaits an interrupt. If it is adequate, the thread that decides the size is adequate interrupts all other threads that have been added and they execute in what is as close as I could get to exact sync.

    I don't think this is the most elegant solution, but unfortunately it is the best I could come up with given my knowledge of Java multi-threading. If anyone can think of a better solution, please let me know. Any feedback is appreciated.

    Thanks!

    submitted by /u/THEBIGSASHA
    [link] [comments]

    What's Local windows debugger in VS?

    Posted: 05 Jul 2020 08:09 PM PDT

    'out of' logic

    Posted: 05 Jul 2020 07:20 PM PDT

    Hi,

    I am programming some tools in tradingview.com to see financial data. If i am looking to build a tool that would flag an event that occurred x out of y days, (ex 7 out of 9 days), how wold I do that? In the past if I looked at 2 or 3 day combos, I would just manually enter all possible combinations but obviously this isn't feasible as the options increase.

    if you are unfamiliar with their "pinescript", just a nudge in the direction of the general programming format would be great and I'll take it from there

    I appreciate any help!

    submitted by /u/edge0785
    [link] [comments]

    looking for help to make a quick little budgeting program

    Posted: 05 Jul 2020 02:56 PM PDT

    hey everyone I've had lots of help here before, and I was wondering if you guys could help again.

    I'm currently trying to write a program in c sharp using visual studios, that is very specific to me to help me quickly work out my budget for the week but id like it to be very customizable

    below is an example of the information id like to receive id like to be asked for how much I've been paid for this week and for it to spit out this result,

    however, id like to be able to change the percentages going into each of the accounts and the amount going into the joint account

    any help would be greatly appreciated I'm a little out of practice since college closed 4 months ago

    £200 paid in a week

    £50 goes to the joint account

    £150 remaining

    75% (£112) goes into my savings accounts

    Halifax Account - 50% (£62.5)

    Cash ISA -50% (£62.5)

    General spending - 25% (£37.5)

    submitted by /u/GenericUser104
    [link] [comments]

    Where is the library for the link() function on macOS?

    Posted: 05 Jul 2020 05:15 PM PDT

    I am currently implementing file system link functions in my cross platform .NET Core program. I'm creating links by P/Invoking the native functions.

    On linux and mac, the function is unistd.h/link(const char *p1, const char *p2). I have tested it in simple C programs on both operating systems, and it works as expected.

    The problem is, that in order to P/Invoke it from C#, I need to know the name of the library file. On linux, the function is in "libc.so.6". What is the equivalent library's name on macOS?

    Here is the apple developer man page for the function, but it doesn't seem to mention the library file.

    submitted by /u/Sparky2199
    [link] [comments]

    Following a flask tutorial, pip is not working and is not properly installed neither is virtualenv or setuptools?

    Posted: 05 Jul 2020 04:04 PM PDT

    Hi all. I'm following this tutorial and I am stuck at the virtualenv part. When I run it, it says command not found. I initially installed it as per the python documentation.

    When I tried to install it via python3 -m pip install —user virtualenv it returns an error "The 'zipp>=0.4' distribution was not found and is required by import lib-resources".

    I have no idea what that means. I read some random github thread that was the only thing I found and it said that I probably have everything installed incorrectly (pip, python, etc) and to uninstall then reinstall. I tried to uninstall pop by tuning pip3 uninstall pip and I have no idea if it worked.

    When I run pip —version I get "Command not found: pip"

    When I run pip3 —version I get: "no module named 'pip._internal.utils.setuptools_build`"

    I've tried updated, reinstalling setuptools, etc etc. What am I doing wrong? Also I am running OSX.

    EDIT: HERE IS COME COMMAND LINE OUTPUT MacBook-Pro ~ % python3 -m ensurepip Looking in links: /var/folders/p5/qd9dt8hj1lqgj9mmw99bcpsm0000gn/T/tmpoztdrlvb Requirement already satisfied: setuptools in /Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages (49.1.0) Requirement already satisfied: pip in /Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages (20.1.1)

    MacBook-Pro ~ % python3 -m pip --version pip 20.1.1 from /Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pip (python 3.8) MacBook-Pro ~ % pip3 -m pip Traceback (most recent call last): File "/Library/Developer/CommandLineTools/usr/bin/pip3", line 6, in <module> from pip._internal import main File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/site-packages/pip/_internal/__init__.py", line 40, in <module> from pip._internal.cli.autocompletion import autocomplete File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/site-packages/pip/_internal/cli/autocompletion.py", line 8, in <module> from pip._internal.cli.main_parser import create_main_parser File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/site-packages/pip/_internal/cli/main_parser.py", line 12, in <module> from pip._internal.commands import ( File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/site-packages/pip/_internal/commands/__init__.py", line 6, in <module> from pip._internal.commands.completion import CompletionCommand File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/site-packages/pip/_internal/commands/completion.py", line 6, in <module> from pip._internal.cli.base_command import Command File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/site-packages/pip/_internal/cli/base_command.py", line 20, in <module> from pip._internal.download import PipSession File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/site-packages/pip/_internal/download.py", line 44, in <module> from pip._internal.utils.setuptools_build import SETUPTOOLS_SHIM ModuleNotFoundError: No module named 'pip._internal.utils.setuptools_build' MacBook-Pro ~ % python3 -m pip Usage: /usr/local/bin/python3 -m pip <command> [options] Commands: install Install packages. download Download packages. uninstall Uninstall packages. freeze Output installed packages in requirements format. list List installed packages. show Show information about installed packages. check Verify installed packages have compatible dependencies. config Manage local and global configuration. search Search PyPI for packages. cache Inspect and manage pip's wheel cache. wheel Build wheels from your requirements. hash Compute hashes of package archives. completion A helper command used for command completion. debug Show information useful for debugging. help Show help for commands. General Options: -h, --help Show help. --isolated Run pip in an isolated mode, ignoring environment variables and user configuration. -v, --verbose Give more output. Option is additive, and can be used up to 3 times. -V, --version Show version and exit. -q, --quiet Give less output. Option is additive, and can be used up to 3 times (corresponding to WARNING, ERROR, and CRITICAL logging levels). --log <path> Path to a verbose appending log. --proxy <proxy> Specify a proxy in the form [user:passwd@]proxy.server:port. --retries <retries> Maximum number of retries each connection should attempt (default 5 times). --timeout <sec> Set the socket timeout (default 15 seconds). --exists-action <action> Default action when a path already exists: (s)witch, (i)gnore, (w)ipe, (b)ackup, (a)bort. --trusted-host <hostname> Mark this host or host:port pair as trusted, even though it does not have valid or any HTTPS. --cert <path> Path to alternate CA bundle. --client-cert <path> Path to SSL client certificate, a single file containing the private key and the certificate in PEM format. --cache-dir <dir> Store the cache data in <dir>. --no-cache-dir Disable the cache. --disable-pip-version-check Don't periodically check PyPI to determine whether a new version of pip is available for download. Implied with --no-index. --no-color Suppress colored output --no-python-version-warning Silence deprecation warnings for upcoming unsupported Pythons. 
    submitted by /u/spiritewolf
    [link] [comments]

    What do you think of my newly released and open sourced Visual Studio Code Theme - Midnight Spruce Pine ?

    Posted: 05 Jul 2020 04:00 PM PDT

    I just released and open sourced my Visual Studio Code Theme: Midnight Spruce Pine !

     

    You can download it directly from Marketplace or search and install the extension/theme from inside Code. You can also download the source on GitHub.

     

    Enjoy!

     

    donate

    submitted by /u/jasonhartsoe
    [link] [comments]

    2.4 ghz mouse adapter.

    Posted: 05 Jul 2020 03:58 PM PDT

    Alright, so to start off, I know practically nothing of any type of coding or programming (if they're even different things, which is what I believe). Regardless of that I am willing to learn and was wondering if there is any way to make a USB stick into an adapter for my mouse. Is there any coding needed for that or would that require an entirely different object? Would I have to mess around with the stick itself such as soldering or anything along those lines?

    Anything would be helpful, thank you and have a great day, evening or night.

    submitted by /u/NeedHelpBluetooth
    [link] [comments]

    How can I make a Blutooth button that turns on a light?

    Posted: 05 Jul 2020 02:24 PM PDT

    So first it's super hard to find information on the proper Bluetooth module to use. Would I just need to figure out the range of the transceiver and receiver and compare it to what I need?

    Also would the language required to write the program be based on which module I choose?

    I'd like to avoid using anything like an arduino/raspberry since I'm trying to keep the cost down on it.

    Basically I want to be able to press a button and turn on a portable light.

    submitted by /u/Badgicthegathering
    [link] [comments]

    Direct eBay Integration API

    Posted: 05 Jul 2020 01:05 PM PDT

    Hey all,

    I don't know if this is the right place to post, if not please advise where's best!

    So, essentially I run my own T-shirt printing business where I print for myself and for others and send direct to their customers.

    I'd like to find someone or a business to help em develop my own software. The back end that I would access would pull all orders from my shopify. Etsy, ebay and amazon store into one place. So as soon as someone orders something from any of those platforms it pulls the order through from platform into my own software so it puts all orders in chronological order wherever they come from.

    I can then print, pack and ship them out and then I hit a button in my software and it'll auto update the relevant platform and therefore notify the end customer.

    I'd eventually love to be able to integrate my software into other people's own accounts on those platforms so Customers could upload their designs to those platforms, as soon as an order comes in it comes straight to my software and I can print pack and ship direct to their customers too.

    I know this sounds like a hell of a lot of work and I don't even know if it's possible, but that's where I hope you can help.

    Does anyone know if this is possible? If so, where would I start looking to find people to work on this project with? And I know "how longs a piece of string" but am I talking $1,000, $5,000, $10,000 or more to produce, would you roughly estimate?

    Thanks for any possible advice!

    submitted by /u/anonymousmice
    [link] [comments]

    VS Code or Visual Studio 2015? VS 2019 is very slow on my old PC.

    Posted: 05 Jul 2020 12:38 PM PDT

    Eventually I will be using Visual Studio 2019 when I get a newer PC but what's my best option for now VS Code or Visual Studio 2015 in terms of easier future transition to VS 2019? Thanks!

    submitted by /u/maxkostikov
    [link] [comments]

    ORM but with a RESTful API underneath instead of a Database?

    Posted: 05 Jul 2020 08:43 AM PDT

    I have a large, hierarchical REST API that I want to interact with. It has dozens of resources, supporting GET, POST, PUT, DELETE for just about every one of them, and each one can have sub-elements, e.g.

    • /users
    • /users/<id>
    • /users/posts
    • /users/posts/<id> etc., etc.,

    Right now I just have a MASSIVE Python class whose constructors creates a session and logs in, and then I have a separate method for every action I can take, e.g.

    • def get_list_of_posts
    • def get_a_post
    • def create_post
    • def get_comments
    • def create_comment and so on,

    The file is so large that my IDE (Pycharm) is freezing with simple edits. So I want to split this into several files. However I would still like them all to share the same session. That way I can use them in a similar way to:

    from media_app import user_session session = user_session(<auth credentials>) my_posts = session.get_list_of_posts(<user_id>) 

    Is there a good design pattern to accomplish something like this? I realize it's very similar to what I would do with an ORM, like

    post = Posts().filter_by(id=<post_id>) 

    I'm wondering if there is a similar technology that wraps over a large API instead of a database.

    submitted by /u/ademwanderer
    [link] [comments]

    Best/classic books?

    Posted: 05 Jul 2020 04:55 AM PDT

    A "friend" on Facebook posted yesterday that he'd been cleaning and disposed of his copy of K&R The C Programming Language(!) My copy disappeared years ago (lent to a colleague...) so I'm thinking I should get a copy to replace it -- even if I haven't written a line of C in decades.

    Any other books that I just have to get?

    submitted by /u/cthart
    [link] [comments]

    How to uninstall Qt Creator completely?

    Posted: 05 Jul 2020 11:32 AM PDT

    I installed Qt Creator, it somehow took up 25GB of storage, how can I uninstall it completely? (on mac OS)

    submitted by /u/cubessssss
    [link] [comments]

    I have access to two 10MBPS internet connections over wifi from 2 different internet providers; how do I use both to download at 20 MBPS ? Does it not work that way?

    Posted: 05 Jul 2020 04:54 AM PDT

    When I try to connect to a network the previous network disconnects. How do I use both connections at the same time?

    submitted by /u/pleaaseeeno92
    [link] [comments]

    Question regarding Open-Source and Licenses

    Posted: 05 Jul 2020 02:59 AM PDT

    I'd like to use different open-source projects as building blocks for a personal project. Some of these open-source projects have the GPL license, and some others have the MIT license.

    I am not really informed when it comes to licensing, so what should I do in case I'd like to publish my own project ? Should I license my project under both the GPL and the MIT license ?

    submitted by /u/OdeToDeath
    [link] [comments]

    What's the best, non-bloated way to get a cross-platform file open dialog, and preferably drop-down menus in a C + SDL2 project?

    Posted: 05 Jul 2020 10:17 AM PDT

    Should I just use some defines to either use the Win32, Xorg, and whatever they call the MacOS interface APIs to make drop-down menus for each major platform, or is there a reasonable cross-platform method that isn't going to load my project up with a thousand dependencies?

    It's for an emulator project, I need to be able to select floppy images on the fly as well as change other various options...

    I've done some Googling on the topic, but can't really seem to find a reasonable, elegant solution. Maybe one doesn't exist.

    Thanks for any input!

    submitted by /u/UselessSoftware
    [link] [comments]

    Compare 2 dictionaries and return missing values

    Posted: 05 Jul 2020 06:02 AM PDT

    Hi there,

    How would I compare these 2 dictionaries and return only the values missing?

    The GetFileListFromBlob(), function gets all file names and I'd like to know what is missing from the db.

    Or is there a better way to get the missing values from these objects?

    Dictionary<int, string> databaseFileList = new Dictionary<int, string>; Dictionary<int, string> blobFileList = new Dictionary<int, string>; int counter = 0; foreach (string f in GetFileListFromDB()) { counter++; databaseFileList.Add(counter, f ); } counter = 0; foreach (string f in GetFileListFromBlob()) { counter++; blobFileList.Add(counter, f); } // How to compare? 

    Thank you

    submitted by /u/greenSacrifice
    [link] [comments]

    Can you explain me this statement "Each object can have different data, but all objects of a class have the same types of data and all objects in a class have the same method."?

    Posted: 04 Jul 2020 11:32 PM PDT

    Kindly explain to me this with the simplest way or easiest example you.

    Thank you!

    submitted by /u/Actual_Camp_f_me
    [link] [comments]

    Strategies for loading HTTP resources in JavaScript?

    Posted: 05 Jul 2020 07:36 AM PDT

    I'm writing a webpage which uses WebGL, and needs to fetch resources to run (shaders, .obj files etc).

    I'm having some trouble doing this in JavaScript though. By using the fetch API and calling it when and where I need the resource, I can't seem to get my code to not get null, as the HTTP request hasn't resolved yet.

    I wanted to know:

    1. How can I be sure that when I need the variable in the code, which is retrieved via a GET request, that it isn't null (unless an error occurred)?
    2. Should I get all the shaders and obj files etc. at the start of the program? Or only when and where needed?

    Thanks!

    submitted by /u/Forumpy
    [link] [comments]

    [Serious] Language communities and BAME support

    Posted: 05 Jul 2020 07:33 AM PDT

    I have a question for the programming community, I hope it is ok to ask it here as I am not sure where else can I do so. I am sorry for anygrief it may cause, but I need guidance.

    I am a girl, BAME, not native english speaker. Finishing studies next year, if COVID lets us.

    I had a chat yesterday with some colleagues about some programming languages I want to learn, and they mentioned how I should avoid them.The reason they gave me was that certain prominent members of that community were known white supremacists and pro-MRA.

    That would make a very unsafe environment for me. I don't mention the languages as I don't want a flame-war about them.I know I may sound naive to many of you, but as you can see I haven't been paying attention

    My question, or questions:

    - are there any language communities known for negative behaviour towards BAME/women which I should avoid?

    - on these communities, are there particularly notorious people at the center of these problems? I ask not to finger-point, but I expect people to move between programming languages, and knowing whom to avoid has saved me in the past

    - any language communities well know for the opposite reason, as equal opportunity and disavowing/expelling white-supremacists or MRA?

    Excuse the throwaway account, but I have too much personal experience with harassment because of my sex and skin. I just want to feel safe.

    submitted by /u/throwawaypigivemetau
    [link] [comments]

    What web based editor is this?

    Posted: 05 Jul 2020 02:39 AM PDT

    I watched a demo of someone editing a progressive web app using this editor in chrome and was wondering if anyone can tell me what the project that made it is called?

    https://1drv.ms/u/s!ArSGQhhyH-dzho57aw204fjp0gyR-A?e=GRL4xb screenshot

    submitted by /u/pyrax69
    [link] [comments]

    No comments:

    Post a Comment