How to create a python script to take information from a public website and put it into excel. Ask Programming |
- How to create a python script to take information from a public website and put it into excel.
- Python/sqlachemy: A value is required for bind parameter 'strip_icc'
- Being asked to build a REST API as a coding test is a regular thing?
- Hiring someone to help me out with a project for my recently incorporated startup . Should I share the whole code?
- It is feasible for someone (who don't work for the community) to reach your IP from a Facebook or Reddit account?
- Get data from api using PHP
- How do i open google.com in android webview using an html file ?
- Can I use NodeJS for API endpoints then Python to parse large datasets?
- What type of thread scheduler is in Python?
- I'm at a loss
- Any ideas to make a neural network prototype in unity or python with easy to gather data or an unsupervised learning?
- Slicing Tiny Desk Concerts based on Youtube Comment Timestamps.
- How can i open urls in webview without using an app ?
- Exiting program after something failed.
- Why does my 2 minute countdown timer not work? (JS and HTML)
- Help in building an advertisement auction
- Should this be left as 3 methods or combined into one? Does it matter?
- What lightweight programming language have you used?
- access timer1.Start() from another class in C#
- Why doesn't TCHAR work for me here? And should I just stop using TCHAR all together?
- Understanding data Structures : Convert hashcode to array index
- Automated Timer Script
- Adding Parameters to Commands using C#
- How to write a parquet bytes object as zipfile to disk
- TCP socket in C++ reads fine from Firefox, but "connection reset by peer" in any other browser
| How to create a python script to take information from a public website and put it into excel. Posted: 20 Mar 2020 12:43 PM PDT Sorry I am new at this and am just looking for information where to look. I have to write reports using the information from a public website. Each entry requires clicking through each tab and searching for the relevant information. It's essentially a number and a year but if it's not the current year than we don't use the number. As in if the test was in 2018 and not being conducted again for three years than we just use the 2018 data. Where could I find more information that could search the website and pull that data into an excel sheet to cut back the amount of time required to search the whole website? Is there also a way to dedicate which cells the information goes into because the names of the information are important? [link] [comments]  |
| Python/sqlachemy: A value is required for bind parameter 'strip_icc' Posted: 20 Mar 2020 07:22 PM PDT upon doing an insertion in python, the console replies back with:
[link] [comments]  |
| Being asked to build a REST API as a coding test is a regular thing? Posted: 20 Mar 2020 08:43 PM PDT This week i started to look for a job as a web developer, i am familiar with Flask, HTML, CSS, JS and Bootstrap4 so i decided to go pro, yesterday i saw a job offer and it looked fine, but they sent me a document with what i needed to do as a test, it was a REST API where an user placed comment and stars(1-5) about a company, the user was able to only watch his comments and the admin was able to see everything. They also asked for a front end to the app and security practices based on OWASP top 10 vuls, they recommend a couple things but i have complete fredom on the language and framework, i can build all that but, is normal that they ask this kind of things? Again, this is my first interview as a web developer. [link] [comments]  |
| Posted: 20 Mar 2020 07:06 PM PDT I have been building a single page app with React and reached the point where I need to have someone more experienced help me with the code (I am a beginner, a few months through). I am a bit worried about sharing the whole code base. At the same time, I don't think this person would be of much help if they didn't have access to everything. Should I just share the whole git repo (with protections such as commiting to master blocked) with this person? Then they'll be able to download it to their machine and forever have the code? What if at some point I no longer wish to work with this person; they'll still have the whole code. Of course, there'll be a contract that forbids them from using the idea. But is this really how things are done? Please advise a scared beginner. [link] [comments]  |
| Posted: 20 Mar 2020 01:14 AM PDT |
| Posted: 20 Mar 2020 06:32 PM PDT I have a task of getting data from an api using PHP. It uses **POST** request, which also sends json body. There is not mentioning of setting any headers, which confused me. I dealt with api's using ajax before, never php. And since json goes both ways, to api and as a response from api, i'm a bit confused. I tried doing it vanilla php way, but on various posts on SO, it is said it's not recommendable. Many have said that using **cURL** is correct way of doing it, so i'm guessing, it's the correct way without involving third party libraries which i(also) cannot use in this case. My code so far: Json body: ``` $str = '{"username":"mymail@mymail.com","password":"blah"}'; //or $arr = [ "username" => ["mymail@mymail.com](mailto:"mymail@mymail.com)", "password" => "blah" ]; ``` function post1($url,$data){ $ch = curl_init(); //Example1 curl_setopt($ch, CURLOPT_HEADER, false); $query = http_build_query($data); //curl_setopt($ch, CURLOPT_HTTPHEADER, ["Content-Type" => "application/json"]); //Alt header for Example2 //curl_setopt($ch, CURLOPT_HTTPHEADER, array("Content-Type: application/json")); $query = $data; curl_setopt($ch,CURLOPT_URL, $url); curl_setopt($ch,CURLOPT_POST, true); curl_setopt($ch,CURLOPT_POSTFIELDS, $query); curl_setopt($ch,CURLOPT_RETURNTRANSFER, true); $response = curl_exec($ch); curl_close($ch); return $response; } //Example1 //echo post1($url1,$str); //VS //Example2 echo post1($url1,$arr); Both json and non-json example produce the same error: ``` {"type":"https://tools.ietf.org/html/rfc7231#section-6.5.13","title":"Unsupported Media Type","status":415,"traceId":"80002328-0002-ff00-b63f-84710c7967bb"} ``` If i use alternative header for example2, it produces different one: ``` {"errors":{"":["Input string '--------------------------7abe186689ac2978' is not a valid number. Path '', line 1, position 42."]},"title":"One or more validation errors occurred.","status":400,"traceId":"8000232a-0002-ff00-b63f-84710c7967bb"} ``` In api document that i have, there is no mention of setting any headers, nonetheless, i tried even setting headers up. Both ways didn't work. Just throws an error. So either i'm missing something in cURL, or they sent me incomplete documentation. Either way, i'm not sure since i never used cURL, nor i ever used this api. I also tried it in Postman, with headers and without, i even tried setting json body with quotation marks(it's silly, i know) just to eliminate whatever possibilities. None of it worked. Mucho kudos in advance to any good people who can present me with solution based on previously stated or at least pointed me in the right direction. [link] [comments]  |
| How do i open google.com in android webview using an html file ? Posted: 20 Mar 2020 05:28 PM PDT |
| Can I use NodeJS for API endpoints then Python to parse large datasets? Posted: 20 Mar 2020 05:26 PM PDT I know Python has decent API libraries like Flask and others, but I'd like to build a website or service where I use Node JS for the API and then any large dataset that needs parsing or any heavy statistical analysis, hand that off to Python to do so. Is this practical and does anyone have any experience handling HTTP requests one way and then handling data processing to Python scripts? [link] [comments]  |
| What type of thread scheduler is in Python? Posted: 20 Mar 2020 05:14 PM PDT I read that a long time ago, Python uses a non-preemptive round robin scheduler, but I can't find sources on it ATM, is this true? [link] [comments]  |
| Posted: 20 Mar 2020 05:04 PM PDT I want to make the jump to a career in programming but I feel like I dont know the steps needed to start that career. I have a bunch of half projects finished from past game developments but nothing as far as applications go. I am 24 and I am afraid that if I dont get this ball rolling this year, then it may never start. Any advice would be helpful, thanks in advance. [link] [comments]  |
| Posted: 20 Mar 2020 04:38 PM PDT So in Catalonia, Spain before accessing university we have to do a TDR (Treball de Recerca) what in English you would call an investigation project, basically we have to look for a theme we're really interested in and make a kind of research paper explaining how it works in-depth and sometimes make a prototype. It's a really important mark of all the two last years of high school and if I do an impressive project I have the opportunity to submit it to some contests and have my research paper posted publicly on google along with my name. The thing is that I choose to investigate Artificial Neural Networks and now I'm on a very early stage of the project where I have to think about what to do as the prototype, I've searched a bit about the subject and know the basics, that there are different learning methods and kinds of neural networks, that there are ways of training the program with data and some times you can just make the program learn by itself with little to no data. It's worth noting that:
that's all, thanks in advance to all of you, it's my first time posting on this sub so if it doesn't fit here please tell me and if possible in which other sub this would fit. [link] [comments]  |
| Slicing Tiny Desk Concerts based on Youtube Comment Timestamps. Posted: 20 Mar 2020 04:31 PM PDT I love tiny desk concerts and want a way to hoard them offline, but I want to split individual songs from the concerts and separate the banter. I was guessing that there could be a way to crawl youtube comments for timestamps .. because usually, people are kind enough to mark them. So I needed help specifically with youtube-comment crawling and splitting audio based on timestamps ... I found a GitHub project that does this but it's based on manual input .. I'm guessing this way of finding timestamps would accelerate the process. [link] [comments]  |
| How can i open urls in webview without using an app ? Posted: 20 Mar 2020 03:40 PM PDT Google Chrome can't load any pages after searching because it's restricted by knox. [link] [comments]  |
| Exiting program after something failed. Posted: 20 Mar 2020 06:20 AM PDT I know it's bad to use functions like exit() etc.. because they don't clean up the stack and deallocate memory. The only good solution is to But before I can go back to main, I need to cleanup by closing a But if I do that, I will lose the error code from If I print the error before I close the handle. Then when I throw the exception to get back to main, and after I have caught that exception in main, I need to use the exception to do something, or else the compilar will keep giving me a warning about unreferenced local variable. But I have already printed the error message before throwing myself back to main. So what do I do then? Print it twice? Ignore the warning? [link] [comments]  |
| Why does my 2 minute countdown timer not work? (JS and HTML) Posted: 20 Mar 2020 02:53 PM PDT When I click the "start" or "reset" button in my browser, nothing happens Does anyone know what I did wrong? HTML <body> <header> <p class="title">Whac-a-mole</p> </header> <section class="game"> <div class="cursor"></div> <img class="mole" src="images/red.png" alt="mole"> </section> <div class="bottom"> <section class="tracker"> <div class="Score">Score</div> <div id="scoreCounter">0</div> </section> <section class="countdown"> <p id="timer">2:00 </p> <button id="startTimer">Start</button> <button id="resetTimer">Reset</button> </section> </div> <script src="spel.js"></script> </body> JavaScript var interval; function countdown () { clearInterval(interval); interval = setInterval( function( ) { var timer = document.getElementById('timer'); timer = timer.split(':'); var mins = [0]; var secs = [1]; secs -= 1; if (mins < 0) return; else if (secs < 0 && mins != 0) { mins -= 1; secs = 59; } else if ( secs < 10 && length.secs != 2) { secs = '0' + secs; } if (mins == 0 && secs == 0) clearInterval(interval); }, 1000); var startTimer = document.getElementById('startTimer'); startTimer.click(function (){ timer.text("2:00"); countdown (); }); var resetTimer = document.getElementById('resetTimer'); resetTimer.click(function (){ timer.text("2:00"); }); } [link] [comments]  |
| Help in building an advertisement auction Posted: 20 Mar 2020 02:38 PM PDT Hey all, I am in the design phase of a social media platform. I would like to include my own ad auction for my site so i have better control of what is displayed, but I am having difficulty trying to figure out how it would work out. I've been researching both Facebook and Google's options so I have a basic idea of how its supposed to work, but I'm not sure how i'd implement something similar for my site. If this were just a simple auction like eBay where people bid for one particular item, and the site calculates the highest bidder, it would be simple. But when working with Ad blocks, people are putting prices on 1000 impressions. (I'm assuming this could be more if the opportunity exist) So when you've got to calculate potentially several hundred or thousands of potential ads, for each of the million plus impression opportunities that a site might get, how do Facebook/ Google or my site determine how much of a budget to devote to each individual auction? With eBay, you know what you are going to get. With ad auctions (to make it simple) you could have 1 eligible auction your campaign qualifies for, or you could have 1000. (yes i know these numbers will be higher, but for simplicity i'm using this example) so if a user's max budget it $5 a day, you wouldn't want to bid all $5 on one impression if there may be 1000 future impressions that you'd also qualify for. So that $5 would need to be split among an unknown X number of auctions. Do Facebook and Google keep track of the winning amount of each individual auction, or do they simply count the number of wins per day and provide an average for their CPC/ CPV rates? Also, since each auction has many different advertisers competing for that impression, each with varying criteria that they selected, how do these auctions quickly match up the demographics of the end user, with the criteria to display a match most likely to be viewed? Is this something a relational database can handle, or should i explore NoSQL databases? (Likely a graph DB) Can the matching be done within the database itself, or would countless queries have to be run for every single option. Millions of these comparisons and auctions would be happening at once, so i'm struggling to figure out the best solution to this. Is there a better language/ framework/ library suited for this application? Is there some code out there already that handles this that can be customized for my sites requirements? If anyone has insight on this or links to resources, it would be appreciated. Thanks [link] [comments]  |
| Should this be left as 3 methods or combined into one? Does it matter? Posted: 20 Mar 2020 02:00 PM PDT I have a program that models an interface for a store. I have - bool checkUser(String s) { //checks } bool checkItem(String s) { //checks other stuff } bool checkPassword(String s) { //more checks } Should I leave that as is, or would it be considered better at all to do like... bool checkThings(String tocheck, String thing) { if(thing == user) ... if (thing == item)... and so on. Thanks for any help, I'm pretty new in any kind of formal education/training [link] [comments]  |
| What lightweight programming language have you used? Posted: 20 Mar 2020 10:05 AM PDT I am interested in small programming languages, using "small" in the vague sense with respect to the size of the compiler, the size of the resulting binary, the amount of syntax in the language, the memory used at runtime, etc. Examples include Lua, Forth, Squirrel, and more as listed on the Wikipedia page for lightweight programming languages Im curious about how much practical application they have received in the general programming community. Have you or your company used them for a project? Did it help make the project successful? Why did you choose it? Im also curious about the domain these kinds of languages are best suited for. For example, would you ever use REBOL for embedded applications, or a webserver? I get that there are some commercial examples of using these languages, like World of Warcraft has used Lua IIRC, but those tend to be the exception rather than the rule---right? What is your commercial experience with these languages, and what did you like/not like about using them? Did using a "lightweight" language give any measurable advantage? Cheers [link] [comments]  |
| access timer1.Start() from another class in C# Posted: 20 Mar 2020 09:17 AM PDT
now i want to know how to access timer1.Start() and timer1.Stop(); in the switch-cases of MyMessageFilter private void timer1_Tick(object sender, EventArgs e) { if (isStartable == true) { mouseClick(); } } class MyMessageFilter : IMessageFilter { public bool PreFilterMessage(ref Message m) { const int WM_LBUTTONDOWN = 0x0201, WM_LBUTTONUP = 0x0202; switch (m.Msg) { case WM_LBUTTONDOWN: // System.Diagnostics.Debug.WriteLine("Left button down " + DateTime.Now); ~> here timer1.Start(); return false; case WM_LBUTTONUP: // System.Diagnostics.Debug.WriteLine("Left button up " + DateTime.Now); ~> here timer1 Stop(); return false; } return false; } } [link] [comments]  |
| Why doesn't TCHAR work for me here? And should I just stop using TCHAR all together? Posted: 20 Mar 2020 03:13 AM PDT I copied the code from this page: https://docs.microsoft.com/en-us/windows/win32/toolhelp/taking-a-snapshot-and-viewing-processes And it gives an error for this line: Error: argument of type "const wchar_t *" is incompatible with parameter of type "TCHAR *" The declaration for printError: Why doesn't TCHAR work for me here? How can I get it to work with TCHAR? What do you think about if I simply stop using TCHAR all together? Because my programs are only for windows. Maybe it will be easier to code by using the unicode versions of all functions etc and adding the 'L' before any literal strings. [link] [comments]  |
| Understanding data Structures : Convert hashcode to array index Posted: 20 Mar 2020 05:23 AM PDT I am trying to understand and implement a hashtable. Specifically the part where the hashcode is converted to an array index. Ex: For a key, say "reddit" , and its value "WEBSITE". The key is passed to a hashing function which returns a Hashcode, an integer. This hashcode is then converted to an array index, and the value is stored as $arr [index] = "WEBSITE" From what I have seen, the hashcode is converted to index using the length of the array. Maybe something like , index = hashcode % arraylength . This is fine. But what happens with a growing array size ? If the array length is used in the calculation of the index. Wouldn't the same hashcode return different index once array length changes ? Only 2 solutions come to mind. Either all indexes are recomputed when array length changes OR some other calculation is used that doesn't use array length, but that has a number of issues. [link] [comments]  |
| Posted: 20 Mar 2020 11:39 AM PDT Hello! I would like to create a python script for a project - I would like the script to trigger a relay I have pre connected, at specific times every day. The light should be on from 9am, until 9pm. The script needs to be able to pick back up if for any reason the Pi was shut off, e.g in a powercut, and resume where it was. I will also obviously make the script run on startup. Can anyone help me with this? Many thanks - Miles [link] [comments]  |
| Adding Parameters to Commands using C# Posted: 20 Mar 2020 11:26 AM PDT I am trying to follow best practices when inserting data into a database, but I am also new to C#. I was following this tutorial, but I found out that when the data gets inserted, I get the "@City" inserted, instead of the actual value. Can anyone help point where or what I am doing wrong? public int addToTable( string data1, string data2) { SqlParameter firstName = new SqlParameter(); SqlParameter lastName = new SqlParameter(); string sqlCommand = "INSERT INTO my_tbl VALUES('@FirstName','@LastName')"; using (SqlConnection sqlConnection = new SqlConnection(connectionString)) { SqlCommand qryCommand = new SqlCommand(sqlCommand, sqlConnection); firstName.ParameterName = "@FirstName"; firstName.Value = data1; lastName.ParameterName = "@LastName"; lastName.Value = data2; qryCommand.Parameters.Add(firstName); qryCommand.Parameters.Add(lastName); try { return qryCommand.ExecuteNonQuery(); } catch (Exception ex){ Console.WriteLine(ex.Message); } } return -1; } [link] [comments]  |
| How to write a parquet bytes object as zipfile to disk Posted: 20 Mar 2020 06:35 AM PDT Hi guys :) I start with a pandas dataframe and I want to save that as a zipped parquet file, all in memory without intermediate steps on the disk. I have the following: But I get this encoding error: "ValueError: stat: embedded null character in path". I got my infos from the only link I found on creating zipfiles from within the memory: https://www.neilgrogan.com/py-bin-zip/ Thank your for your help :) [link] [comments]  |
| TCP socket in C++ reads fine from Firefox, but "connection reset by peer" in any other browser Posted: 20 Mar 2020 08:59 AM PDT An intern wrote a web server on Linux in C++ during the summer, he was brand new to sockets. The server basically works when accessing it from Firefox, but when accessing from Chrome, IE, Edge, Opera - all fail at read(sockHandle, buffer, sizeof(buffer) - 1) : errno 254 , Connection reset by peer. I am brand new to sockets too. I suspect this is has something to do with the browser's settings and the socket's options, but which ones specifically, it at all - I have no idea... // create for standard IPv4 TCP/IP addressing socket(AF_INET, SOCK_STREAM, IPPROTO_TCP) // set the socket to non blocking signal driven sockResult = set_sock_opts(srvHandle); // set the socket reuse option on the server sock // to get rid of unnessary relaunch errors setsockopt(srvHandle, SOL_SOCKET, SO_REUSEADDR, &ENABLE, sizeof(int)); sockaddr_in srvAddr; bzero(&srvAddr, sizeof(sockaddr_in)); srvAddr.sin_family = AF_INET; srvAddr.sin_addr.s_addr = htonl(INADDR_ANY); srvAddr.sin_port = htons(listenPort); bind(srvHandle, (sockaddr*)&srvAddr, sizeof(sockaddr_in)) listen(srvHandle, pendingQ) Then: pollfd pfd; pfd.fd = sockHandle; /* file descriptor */ pfd.events = POLLIN; /* events to look for */ int ret = poll(&pfd, 1, 5000); read(sockHandle, buffer, sizeof(buffer) - 1);
Any suggestion is greatly appreciated! [link] [comments]  |
| You are subscribed to email updates from AskProgramming. To stop receiving these emails, you may unsubscribe now. | Email delivery powered by Google |
| Google, 1600 Amphitheatre Parkway, Mountain View, CA 94043, United States | |
No comments:
Post a Comment